Jul 23 21:51:17.603009 ip-10-0-143-223 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 21:51:18.060885 ip-10-0-143-223 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 21:51:18.060885 ip-10-0-143-223 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 21:51:18.060885 ip-10-0-143-223 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 21:51:18.060885 ip-10-0-143-223 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 21:51:18.060885 ip-10-0-143-223 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 21:51:18.062432 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.062342    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 21:51:18.067943 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067927    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 21:51:18.067943 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067944    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067948    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067951    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067954    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067957    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067960    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067963    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067966    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067970    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067974    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067984    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067988    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067990    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067993    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067996    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.067999    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068001    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068004    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068007    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 21:51:18.068012 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068009    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068013    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068015    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068018    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068021    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068024    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068027    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068030    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068033    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068035    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068038    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068041    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068043    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068047    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068051    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068054    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068056    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068059    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068061    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068063    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 21:51:18.068455 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068066    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068068    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068071    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068077    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068080    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068082    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068085    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068087    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068090    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068092    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068095    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068098    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068102    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068104    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068108    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068110    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068113    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068116    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068119    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 21:51:18.068974 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068122    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068125    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068128    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068130    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068133    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068136    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068138    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068141    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068144    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068146    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068149    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068151    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068154    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068156    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068159    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068163    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068165    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068169    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068171    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068174    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 21:51:18.069464 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068177    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068179    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068181    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068184    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068186    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068189    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068191    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068195    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068197    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068200    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068203    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068205    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068208    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068210    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068213    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068216    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068218    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068221    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068223    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068226    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 21:51:18.070005 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068983    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068990    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068993    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068996    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.068999    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069002    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069006    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069008    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069011    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069013    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069017    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069020    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069023    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069025    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069028    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069031    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069033    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069035    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069038    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 21:51:18.070522 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069041    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069044    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069046    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069049    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069051    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069054    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069056    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069059    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069061    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069064    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069067    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069069    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069072    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069074    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069077    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069079    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069081    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069084    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069086    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069089    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 21:51:18.071062 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069092    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069095    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069097    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069100    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069102    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069105    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069107    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069110    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069113    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069115    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069119    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069121    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069124    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069127    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069129    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069131    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069134    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069136    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069139    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069141    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 21:51:18.071656 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069143    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069146    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069149    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069151    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069154    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069156    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069158    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069161    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069164    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069167    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069170    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069172    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069175    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069178    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069180    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069183    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069185    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069188    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069191    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 21:51:18.072177 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069194    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069197    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069199    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069201    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069204    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069208    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069212    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069215    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069218    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069221    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069224    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069227    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069230    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069233    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069236    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069240    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069244    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069247    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069249    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069252    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 21:51:18.072720 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069255    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069711    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069724    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069730    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069734    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069737    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069740    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069742    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069745    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069748    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069750    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069754    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069756    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069759    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069762    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069764    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069766    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069769    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069772    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 21:51:18.073252 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069774    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069777    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069780    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069782    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069785    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069788    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069790    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069792    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069795    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069798    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069800    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069803    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069805    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069808    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069811    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069817    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069820    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069823    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069826    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069828    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 21:51:18.073738 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069831    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069833    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069837    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069840    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069842    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069844    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069847    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069850    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069852    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069855    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069858    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069860    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069863    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069865    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069868    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069870    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069873    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069875    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069878    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069880    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 21:51:18.074222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069883    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069885    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069888    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069890    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069893    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069895    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069897    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069902    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069905    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069907    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069910    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069912    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069915    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069917    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069921    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069923    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069926    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069928    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069931    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069934    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 21:51:18.074713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069936    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069939    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069941    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069944    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069946    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069949    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069951    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069954    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069956    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069959    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069963    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069966    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069970    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069973    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069975    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069978    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069980    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069984    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069987    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 21:51:18.075205 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069990    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.069993    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070060    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070067    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070073    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070078    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070082    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070085    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070089    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070094    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070097    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070100    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070104    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070107    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070110    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070113    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070116    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070119    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070122    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070125    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070129    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070132    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070135    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070138    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 21:51:18.075673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070142    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070145    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070149    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070152    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070156    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070159    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070165    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070168    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070172    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070175    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070179    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070182    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070185    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070188    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070191    2575 flags.go:64] FLAG: --enable-server="true"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070194    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070199    2575 flags.go:64] FLAG: --event-burst="100"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070202    2575 flags.go:64] FLAG: --event-qps="50"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070205    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070208    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070210    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070214    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070217    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070219    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070222    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 23 21:51:18.076240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070225    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070228    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070231    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070234    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070237    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070240    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070243    2575 flags.go:64] FLAG: --feature-gates=""
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070247    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070250    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070253    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070256    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070259    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070262    2575 flags.go:64] FLAG: --help="false"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070267    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070270    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070273    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070276    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070279    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070283    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070286    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070289    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070292    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070295    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 21:51:18.076845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070298    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070301    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070304    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070307    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070309    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070312    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070315    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070318    2575 flags.go:64] FLAG: --lock-file=""
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070321    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070324    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070326    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070332    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070334    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070337    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070340    2575 flags.go:64] FLAG: --logging-format="text"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070343    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070346    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070349    2575 flags.go:64] FLAG: --manifest-url=""
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070352    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070356    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070359    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070363    2575 flags.go:64] FLAG: --max-pods="110"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070367    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070371    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070374    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 21:51:18.077487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070377    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070380    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070383    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070387    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070394    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070397    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070399    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070402    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070405    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070411    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070413    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070417    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070420    2575 flags.go:64] FLAG: --port="10250"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070423    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070426    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-071f55c6cbcd55eb2"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070429    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070432    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070435    2575 flags.go:64] FLAG: --register-node="true"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070438    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070441    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070444    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070447    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070450    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070454    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070457    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 21:51:18.078304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070460    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070462    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070465    2575 flags.go:64] FLAG: --runonce="false"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070468    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070473    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070476    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070479    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070482    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070485    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070488    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070491    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070494    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070497    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070500    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070503    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070506    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070509    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070511    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070517    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070519    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070523    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070527    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070530    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070533    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070536    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 21:51:18.079175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070539    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070542    2575 flags.go:64] FLAG: --v="2"
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070545    2575 flags.go:64] FLAG: --version="false"
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070549    2575 flags.go:64] FLAG: --vmodule=""
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070553    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.070556    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070696    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070700    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070704    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070707    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070709    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070713    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070716    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070719    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070723    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070725    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070728    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070730    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070733    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070736    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070739    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 21:51:18.079801 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070742    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070744    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070747    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070749    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070752    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070756    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070759    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070762    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070765    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070768    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070770    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070773    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070775    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070778    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070780    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070783    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070786    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070788    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070792    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 21:51:18.080317 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070795    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070798    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070801    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070807    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070810    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070813    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070817    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070820    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070823    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070826    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070828    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070831    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070834    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070837    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070839    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070841    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070844    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070847    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070849    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 21:51:18.080810 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070852    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070854    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070856    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070859    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070862    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070864    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070867    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070869    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070872    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070875    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070877    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070880    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070882    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070885    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070888    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070890    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070894    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070896    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070899    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070902    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 21:51:18.081306 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070905    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070908    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070910    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070913    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070915    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070918    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070921    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070923    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070926    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070928    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070931    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070933    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070936    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070938    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070941    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070943    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070946    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070948    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070951    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070953    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 21:51:18.081806 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070956    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070959    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070961    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070963    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070966    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.070969    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.071625    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.079977    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.079994    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.081290    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.081298    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081359    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081364    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081368    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081371    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 21:51:18.082284 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081374    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081377    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081380    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081383    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081386    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081389    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081392    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081395    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081398    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081400    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081403    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081405    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081408    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081411    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081413    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081416    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081418    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081421    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081425    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081429    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 21:51:18.082782 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081433    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081436    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081440    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081444    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081447    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081451    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081454    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081457    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081460    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081463    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081466    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081469    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081472    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081474    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081477    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081479    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081482    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081484    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081487    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081490    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 21:51:18.083273 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081493    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081495    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081497    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081501    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081503    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081506    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081508    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081511    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081513    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081516    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081518    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081521    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081523    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081526    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081529    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081531    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081534    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081537    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081540    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081543    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 21:51:18.083872 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081545    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081548    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081551    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081553    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081556    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081558    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081562    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081565    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081568    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081571    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081574    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081577    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081580    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081582    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081585    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081594    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081597    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081599    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081602    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 21:51:18.084360 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081604    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081607    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081610    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081612    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081615    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081617    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081620    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081623    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081625    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081641    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081645    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081648    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081650    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081653    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081656    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 21:51:18.084828 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081658    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.081663    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081790    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081794    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081797    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081800    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081803    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081806    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081809    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081812    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081815    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081817    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081820    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081823    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 21:51:18.085187 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081833    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081837    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081840    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081843    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081846    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081849    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081851    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081854    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081856    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081859    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081862    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081864    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081867    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081870    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081873    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081875    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081878    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081880    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081883    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081885    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 21:51:18.085525 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081888    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081890    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081893    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081895    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081898    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081900    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081903    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081905    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081907    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081910    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081912    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081915    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081917    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081925    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081928    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081931    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081933    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081935    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081938    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081940    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 21:51:18.086014 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081943    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081945    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081948    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081950    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081952    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081955    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081958    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081960    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081963    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081965    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081968    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081970    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081973    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081975    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081977    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081980    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081982    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081986    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081990    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081992    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 21:51:18.086576 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081996    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.081998    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082001    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082004    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082008    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082011    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082015    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082018    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082021    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082023    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082026    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082028    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082031    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082034    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082036    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082038    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082041    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082043    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082046    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 21:51:18.087070 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082049    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082051    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082054    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082056    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082059    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082061    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082064    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:18.082066    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.082071    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.082171    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 21:51:18.087521 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.086118    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 21:51:18.087842 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.087644    2575 server.go:1019] "Starting client certificate rotation"
Jul 23 21:51:18.087842 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.087764    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 21:51:18.087842 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.087819    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 21:51:18.090498 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.090480    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 21:51:18.091757 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.091737    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 21:51:18.104925 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.104907    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 23 21:51:18.107167 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.107154    2575 log.go:25] "Validated CRI v1 image API"
Jul 23 21:51:18.107922 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.107909    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 21:51:18.110382 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.110361    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7fc1079b-4e20-4f4e-88c7-0c0858eacc01:/dev/nvme0n1p3 f892c4a7-be5c-40d6-b4ec-fc9c9e25b2ef:/dev/nvme0n1p4]
Jul 23 21:51:18.110456 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.110382    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 21:51:18.118101 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.117985    2575 manager.go:216] Machine: {Timestamp:2026-07-23 21:51:18.116447985 +0000 UTC m=+0.377447704 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088889 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a29db2e621534f6c1e79495857696 SystemUUID:ec2a29db-2e62-1534-f6c1-e79495857696 BootID:341af060-a288-4711-8535-fa233461e280 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3b:e1:e0:c1:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3b:e1:e0:c1:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:83:26:ea:1b:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 21:51:18.118101 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.118092    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 21:51:18.118245 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.118175    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 21:51:18.118527 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.118507    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 21:51:18.120954 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.120930    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 21:51:18.121087 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.120959    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-223.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 21:51:18.121804 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.121794    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 21:51:18.121836 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.121806    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 21:51:18.121836 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.121813    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 21:51:18.122532 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.122522    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 21:51:18.124353 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.124343    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 21:51:18.127758 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127747    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 21:51:18.127879 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127870    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 21:51:18.127911 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127899    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 21:51:18.127911 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127907    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 21:51:18.127961 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127923    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 21:51:18.127961 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127931    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 23 21:51:18.127961 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.127940    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 21:51:18.129120 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.129105    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 21:51:18.129120 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.129131    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 21:51:18.132404 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.132388    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 21:51:18.134311 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.134298    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 21:51:18.134369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.134333    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 21:51:18.135067 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135054    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 21:51:18.135107 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135073    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 21:51:18.135107 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135079    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 21:51:18.135107 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135085    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 21:51:18.135107 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135093    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 21:51:18.135107 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135099    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 21:51:18.135240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135109    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 21:51:18.135240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135119    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 21:51:18.135240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135129    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 21:51:18.135240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135135    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 21:51:18.135240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135145    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 21:51:18.135240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135150    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 21:51:18.136008 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.135997    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 21:51:18.136039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.136012    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 21:51:18.139769 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.139731    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-223.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 21:51:18.139769 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.139762    2575 server.go:1268] "Started kubelet"
Jul 23 21:51:18.139903 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.139877    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 21:51:18.139962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.139884    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 21:51:18.139962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.139931    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 21:51:18.140650 ip-10-0-143-223 systemd[1]: Started Kubernetes Kubelet.
Jul 23 21:51:18.141075 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.140713    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 21:51:18.141075 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.140727    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 21:51:18.141954 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.141936    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 21:51:18.144395 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.143439    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 21:51:18.145923 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.145906    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 21:51:18.148987 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.147904    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-223.ec2.internal.18c509afd62d4b3e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-223.ec2.internal,UID:ip-10-0-143-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-223.ec2.internal,},FirstTimestamp:2026-07-23 21:51:18.139489086 +0000 UTC m=+0.400488805,LastTimestamp:2026-07-23 21:51:18.139489086 +0000 UTC m=+0.400488805,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-223.ec2.internal,}"
Jul 23 21:51:18.151286 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.151260    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 21:51:18.151286 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.151270    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 21:51:18.151490 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.151460    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.152054 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152036    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 21:51:18.152054 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152036    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 21:51:18.152200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152064    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 21:51:18.152200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152098    2575 factory.go:55] Registering systemd factory
Jul 23 21:51:18.152200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152157    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 23 21:51:18.152200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152185    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 21:51:18.152200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152194    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 21:51:18.152444 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152385    2575 factory.go:153] Registering CRI-O factory
Jul 23 21:51:18.152444 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152397    2575 factory.go:223] Registration of the crio container factory successfully
Jul 23 21:51:18.152545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152449    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 21:51:18.152545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152473    2575 factory.go:103] Registering Raw factory
Jul 23 21:51:18.152545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152485    2575 manager.go:1196] Started watching for new ooms in manager
Jul 23 21:51:18.152997 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.152977    2575 manager.go:349] Starting recovery of all containers
Jul 23 21:51:18.155300 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.155124    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 21:51:18.155300 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.155229    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 21:51:18.164921 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.164892    2575 manager.go:354] Recovery completed
Jul 23 21:51:18.165066 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.165029    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2v97"
Jul 23 21:51:18.169491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.169479    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.171221 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171202    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.171277 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171236    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.171277 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171252    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.171815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171799    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 21:51:18.171815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171812    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 21:51:18.171920 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171828    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 21:51:18.171920 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.171813    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.172505 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.172489    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.172560 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.172517    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.172560 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.172527    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.172895 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.172880    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2v97"
Jul 23 21:51:18.173462 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.173389    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-223.ec2.internal.18c509afd81178d8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-223.ec2.internal,UID:ip-10-0-143-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-223.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-223.ec2.internal,},FirstTimestamp:2026-07-23 21:51:18.171220184 +0000 UTC m=+0.432219905,LastTimestamp:2026-07-23 21:51:18.171220184 +0000 UTC m=+0.432219905,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-223.ec2.internal,}"
Jul 23 21:51:18.174821 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.174804    2575 policy_none.go:49] "None policy: Start"
Jul 23 21:51:18.174821 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.174822    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 21:51:18.174965 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.174834    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 21:51:18.175818 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.175807    2575 policy_none.go:47] "Start"
Jul 23 21:51:18.206410 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.206225    2575 manager.go:337] "Starting Device Plugin manager"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.206446    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.206459    2575 server.go:87] "Starting device plugin registration server"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.206724    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.206736    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.207244    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.207332    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.207341    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.207703    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.207754    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.218989    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.220071    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.220104    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.220143    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.220173    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 21:51:18.227831 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.222684    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 21:51:18.308029 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.307999    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.309093 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.309070    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.309214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.309107    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.309214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.309122    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.309214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.309152    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.318416 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.318393    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.318507 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.318422    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-223.ec2.internal\": node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.320963 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.320944    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"]
Jul 23 21:51:18.321052 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.321015    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.321932 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.321911    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.322046 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.321946    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.322046 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.321962    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.324300 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.324281    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.324420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.324403    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.324468 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.324437    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.325021 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.325004    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.325091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.325031    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.325091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.325044    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.325091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.325065    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.325091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.325082    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.325222 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.325094    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.327267 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.327250    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.327419 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.327281    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 21:51:18.328024 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.328005    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 21:51:18.328113 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.328040    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 21:51:18.328113 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.328052    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeHasSufficientPID"
Jul 23 21:51:18.345314 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.345293    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.353455 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.353437    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e82e16c6891169bbedf95a1d89739584-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal\" (UID: \"e82e16c6891169bbedf95a1d89739584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.353541 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.353468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa48ab49f2c0e7a214bbd553b8e1864c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-223.ec2.internal\" (UID: \"aa48ab49f2c0e7a214bbd553b8e1864c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.353541 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.353493    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e82e16c6891169bbedf95a1d89739584-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal\" (UID: \"e82e16c6891169bbedf95a1d89739584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.365363 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.365342    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-223.ec2.internal\" not found" node="ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.369984 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.369969    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-223.ec2.internal\" not found" node="ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.446305 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.446275    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.453811 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.453783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa48ab49f2c0e7a214bbd553b8e1864c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-223.ec2.internal\" (UID: \"aa48ab49f2c0e7a214bbd553b8e1864c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.453889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.453816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e82e16c6891169bbedf95a1d89739584-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal\" (UID: \"e82e16c6891169bbedf95a1d89739584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.453889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.453833    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e82e16c6891169bbedf95a1d89739584-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal\" (UID: \"e82e16c6891169bbedf95a1d89739584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.453889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.453882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e82e16c6891169bbedf95a1d89739584-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal\" (UID: \"e82e16c6891169bbedf95a1d89739584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.454019 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.453871    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa48ab49f2c0e7a214bbd553b8e1864c-config\") pod \"kube-apiserver-proxy-ip-10-0-143-223.ec2.internal\" (UID: \"aa48ab49f2c0e7a214bbd553b8e1864c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.454019 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.453932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e82e16c6891169bbedf95a1d89739584-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal\" (UID: \"e82e16c6891169bbedf95a1d89739584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.547015 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.546949    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.647732 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.647697    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.668306 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.668265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.675378 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:18.675351    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:18.748453 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.748418    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.849065 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.848989    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:18.949255 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:18.949217    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.050177 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.050138    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.088136 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.088097    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 21:51:19.088617 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.088312    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 21:51:19.151021 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.150983    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.178927 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.178883    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 21:46:18 +0000 UTC" deadline="2028-05-07 15:37:09.9127342 +0000 UTC"
Jul 23 21:51:19.178927 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.178920    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15689h45m50.733817938s"
Jul 23 21:51:19.252052 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.252002    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.286223 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:19.286185    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa48ab49f2c0e7a214bbd553b8e1864c.slice/crio-1ac009e148d2b7e6ad84feccde138e1b18c2bf76bdd84ce36e936335fb214095 WatchSource:0}: Error finding container 1ac009e148d2b7e6ad84feccde138e1b18c2bf76bdd84ce36e936335fb214095: Status 404 returned error can't find the container with id 1ac009e148d2b7e6ad84feccde138e1b18c2bf76bdd84ce36e936335fb214095
Jul 23 21:51:19.293762 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.293743    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 21:51:19.294644 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:19.294624    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82e16c6891169bbedf95a1d89739584.slice/crio-1e8770e8c8a29911c8c7af7514aac0a15336952a75cbb3e5632323027e53cd04 WatchSource:0}: Error finding container 1e8770e8c8a29911c8c7af7514aac0a15336952a75cbb3e5632323027e53cd04: Status 404 returned error can't find the container with id 1e8770e8c8a29911c8c7af7514aac0a15336952a75cbb3e5632323027e53cd04
Jul 23 21:51:19.353046 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.352997    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.453481 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.453385    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.553542 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.553504    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.598345 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.598315    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 21:51:19.653836 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:19.653793    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-223.ec2.internal\" not found"
Jul 23 21:51:19.702531 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.702493    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 21:51:19.745331 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.745247    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 21:51:19.752189 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.752153    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:19.766452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.766423    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 21:51:19.767679 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.767658    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal"
Jul 23 21:51:19.776987 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:19.776960    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 21:51:20.129108 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.129031    2575 apiserver.go:52] "Watching apiserver"
Jul 23 21:51:20.137549 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.137525    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 21:51:20.139019 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.138992    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-sftqj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal","openshift-multus/network-metrics-daemon-9t6qd","openshift-ovn-kubernetes/ovnkube-node-wql74","kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8","openshift-cluster-node-tuning-operator/tuned-jlr7n","openshift-image-registry/node-ca-nz8gs","openshift-multus/multus-8bn87","openshift-multus/multus-additional-cni-plugins-vc8qb","openshift-network-diagnostics/network-check-target-tq988","openshift-network-operator/iptables-alerter-rk7ps","kube-system/global-pull-secret-syncer-29qg9","kube-system/konnectivity-agent-mfvsh"]
Jul 23 21:51:20.142249 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.142211    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.145226 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.145202    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.145333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.145250    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.145400 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.145326    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ffn2s\""
Jul 23 21:51:20.145650 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.145632    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 21:51:20.145735 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.145709    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.148004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.147592    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 21:51:20.148004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.147818    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.148004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.147871    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cxbjf\""
Jul 23 21:51:20.148224 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.148060    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 21:51:20.148776 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.148359    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.148776 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.148493    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.148776 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.148361    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 21:51:20.150994 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.150763    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l7dh9\""
Jul 23 21:51:20.150994 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.150766    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 21:51:20.150994 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.150943    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.152525 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.152498    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 21:51:20.153283 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153261    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6c8jk\""
Jul 23 21:51:20.153378 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153340    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 21:51:20.153523 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153489    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 21:51:20.153585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153538    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 21:51:20.153585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153557    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 21:51:20.153585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153566    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.153784 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.153731    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.154139 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.154008    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.156582 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.156107    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.156582 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.156481    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4v5r\""
Jul 23 21:51:20.159879 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.157487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.159879 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.157826    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.160492 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.160061    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.160492 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.160381    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.160869 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.160786    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c7796\""
Jul 23 21:51:20.161850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.161826    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.162825 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.162801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wc5\" (UniqueName: \"kubernetes.io/projected/b7c67823-8460-44f5-a9e1-7aa98e7c7165-kube-api-access-j4wc5\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.162972 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.162927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8b2d5df-b34f-4985-b490-555ffb020fcc-tmp-dir\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.162972 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.162964    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-k8s-cni-cncf-io\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.163105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.162989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-slash\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.163105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.163105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163077    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-cni-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163110    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj49l\" (UniqueName: \"kubernetes.io/projected/d8b2d5df-b34f-4985-b490-555ffb020fcc-kube-api-access-lj49l\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163134    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c44tw\" (UniqueName: \"kubernetes.io/projected/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-kube-api-access-c44tw\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cnibin\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-cnibin\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-hostroot\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163227    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-os-release\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.163304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163250    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4f9f\" (UniqueName: \"kubernetes.io/projected/49b50422-b290-4646-babf-e85f78ea429c-kube-api-access-f4f9f\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163976    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.164142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.163988    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 21:51:20.164142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-var-lib-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8b2d5df-b34f-4985-b490-555ffb020fcc-hosts-file\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.164142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164113    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164157    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164159    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-multus-certs\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-etc-kubernetes\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-cni-bin\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-cni-multus\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164300    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-ovn\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-run-ovn-kubernetes\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-system-cni-dir\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.164441 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49b50422-b290-4646-babf-e85f78ea429c-cni-binary-copy\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164460    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-cni-bin\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-kubelet\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164545    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovn-node-metrics-cert\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-registration-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164598    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-device-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-conf-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-systemd-units\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164714    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-systemd\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164737    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-etc-selinux\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164786    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49b50422-b290-4646-babf-e85f78ea429c-multus-daemon-config\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164807    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-kubelet\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-node-log\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovnkube-config\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.164936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164884    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.164909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-log-socket\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165050    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zbr9\" (UniqueName: \"kubernetes.io/projected/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-kube-api-access-7zbr9\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.165029    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-socket-dir-parent\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-etc-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovnkube-script-lib\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-system-cni-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165236    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-netns\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-cni-netd\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165314    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-socket-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-sys-fs\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-os-release\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165439    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.165668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.166490 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165529    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5twfc\""
Jul 23 21:51:20.166490 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-run-netns\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.166490 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.165636    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-env-overrides\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.167596 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.167572    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:20.167685 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.167658    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:20.168890 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.168861    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 21:51:20.170350 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.170313    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.173333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.172695    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 21:51:20.173333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.172947    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-krv2k\""
Jul 23 21:51:20.173333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.173005    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 21:51:20.173333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.173174    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:51:20.173668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.173544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.173668 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.173619    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:20.175839 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.175817    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.178690 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.178546    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 21:51:20.178690 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.178565    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 21:51:20.178859 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.178725    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9n7p2\""
Jul 23 21:51:20.192592 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.192566    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jzm8v"
Jul 23 21:51:20.201395 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.201373    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jzm8v"
Jul 23 21:51:20.225886 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.225772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal" event={"ID":"e82e16c6891169bbedf95a1d89739584","Type":"ContainerStarted","Data":"1e8770e8c8a29911c8c7af7514aac0a15336952a75cbb3e5632323027e53cd04"}
Jul 23 21:51:20.227694 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.227627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal" event={"ID":"aa48ab49f2c0e7a214bbd553b8e1864c","Type":"ContainerStarted","Data":"1ac009e148d2b7e6ad84feccde138e1b18c2bf76bdd84ce36e936335fb214095"}
Jul 23 21:51:20.253734 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.253666    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 21:51:20.265815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-os-release\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.265971 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4f9f\" (UniqueName: \"kubernetes.io/projected/49b50422-b290-4646-babf-e85f78ea429c-kube-api-access-f4f9f\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.265971 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-var-lib-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.265971 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8b2d5df-b34f-4985-b490-555ffb020fcc-hosts-file\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.265971 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx26l\" (UniqueName: \"kubernetes.io/projected/5cba6262-d5b6-4e09-8b0f-78253daba24d-kube-api-access-wx26l\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.265971 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.265971 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-multus-certs\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-etc-kubernetes\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265982    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-var-lib-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.265995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-os-release\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266001    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysctl-d\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-etc-kubernetes\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266046    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-multus-certs\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxxsn\" (UniqueName: \"kubernetes.io/projected/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-kube-api-access-pxxsn\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266086    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a91e5420-94f1-4941-82f5-1cf1eae0496c-agent-certs\") pod \"konnectivity-agent-mfvsh\" (UID: \"a91e5420-94f1-4941-82f5-1cf1eae0496c\") " pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-cni-bin\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266059    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8b2d5df-b34f-4985-b490-555ffb020fcc-hosts-file\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-cni-multus\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266168    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-ovn\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266180    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-cni-bin\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266191    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-run-ovn-kubernetes\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266228    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkkqf\" (UniqueName: \"kubernetes.io/projected/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-kube-api-access-lkkqf\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.266289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-run-ovn-kubernetes\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-cni-multus\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266280    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-ovn\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-kubernetes\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysctl-conf\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-systemd\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266396    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-system-cni-dir\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49b50422-b290-4646-babf-e85f78ea429c-cni-binary-copy\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266535    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-system-cni-dir\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-cni-bin\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-kubelet\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-cni-bin\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovn-node-metrics-cert\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266685    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-var-lib-kubelet\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266711    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5cba6262-d5b6-4e09-8b0f-78253daba24d-iptables-alerter-script\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266737    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cba6262-d5b6-4e09-8b0f-78253daba24d-host-slash\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-registration-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.267235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-device-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-conf-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-systemd-units\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266863    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-systemd\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266878    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-run\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-var-lib-kubelet\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-device-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-etc-selinux\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266943    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266961    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-registration-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.266971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49b50422-b290-4646-babf-e85f78ea429c-multus-daemon-config\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-kubelet\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267004    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267023    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-conf-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267040    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-node-log\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49b50422-b290-4646-babf-e85f78ea429c-cni-binary-copy\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267091    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-node-log\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovnkube-config\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-systemd-units\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267127    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267156    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhvsv\" (UniqueName: \"kubernetes.io/projected/f6571af9-eab4-4e45-8560-9c0065dcabfd-kube-api-access-hhvsv\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-log-socket\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-etc-selinux\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267212    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-host\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267259    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-serviceca\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-kubelet\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysconfig\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267334    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-sys\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267364    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zbr9\" (UniqueName: \"kubernetes.io/projected/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-kube-api-access-7zbr9\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-systemd\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-socket-dir-parent\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-etc-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-log-socket\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovnkube-script-lib\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-tuned\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267555    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49b50422-b290-4646-babf-e85f78ea429c-multus-daemon-config\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-tmp\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267750    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-etc-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267786    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a91e5420-94f1-4941-82f5-1cf1eae0496c-konnectivity-ca\") pod \"konnectivity-agent-mfvsh\" (UID: \"a91e5420-94f1-4941-82f5-1cf1eae0496c\") " pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-system-cni-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267871    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovnkube-config\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-netns\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.267938    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-cni-netd\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268000    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-system-cni-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-socket-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268070    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-socket-dir-parent\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-sys-fs\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovnkube-script-lib\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.269991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268123    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-os-release\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268148    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-netns\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-cni-netd\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-sys-fs\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268227    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-os-release\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-socket-dir\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/886ede06-fccf-4379-831a-41c244ab7387-dbus\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-run-netns\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-env-overrides\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-run-netns\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268480    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/886ede06-fccf-4379-831a-41c244ab7387-kubelet-config\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wc5\" (UniqueName: \"kubernetes.io/projected/b7c67823-8460-44f5-a9e1-7aa98e7c7165-kube-api-access-j4wc5\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8b2d5df-b34f-4985-b490-555ffb020fcc-tmp-dir\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-k8s-cni-cncf-io\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.270884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-slash\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-cni-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268812    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-host-slash\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj49l\" (UniqueName: \"kubernetes.io/projected/d8b2d5df-b34f-4985-b490-555ffb020fcc-kube-api-access-lj49l\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268845    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8b2d5df-b34f-4985-b490-555ffb020fcc-tmp-dir\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b7c67823-8460-44f5-a9e1-7aa98e7c7165-run-openvswitch\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268960    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c44tw\" (UniqueName: \"kubernetes.io/projected/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-kube-api-access-c44tw\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268887    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-multus-cni-dir\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.268995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-host-run-k8s-cni-cncf-io\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cnibin\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-cnibin\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-hostroot\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.271728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-modprobe-d\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269138    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-cnibin\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-cnibin\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269145    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-lib-modules\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49b50422-b290-4646-babf-e85f78ea429c-hostroot\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269231    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-host\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.269681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b7c67823-8460-44f5-a9e1-7aa98e7c7165-env-overrides\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.272633 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.271611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7c67823-8460-44f5-a9e1-7aa98e7c7165-ovn-node-metrics-cert\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.274428 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.274370    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4f9f\" (UniqueName: \"kubernetes.io/projected/49b50422-b290-4646-babf-e85f78ea429c-kube-api-access-f4f9f\") pod \"multus-8bn87\" (UID: \"49b50422-b290-4646-babf-e85f78ea429c\") " pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.276150 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.276122    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zbr9\" (UniqueName: \"kubernetes.io/projected/5209be5a-03c5-4cf0-b6bc-dbf4acf85df3-kube-api-access-7zbr9\") pod \"multus-additional-cni-plugins-vc8qb\" (UID: \"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3\") " pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.276620 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.276571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj49l\" (UniqueName: \"kubernetes.io/projected/d8b2d5df-b34f-4985-b490-555ffb020fcc-kube-api-access-lj49l\") pod \"node-resolver-sftqj\" (UID: \"d8b2d5df-b34f-4985-b490-555ffb020fcc\") " pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.277065 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.277041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wc5\" (UniqueName: \"kubernetes.io/projected/b7c67823-8460-44f5-a9e1-7aa98e7c7165-kube-api-access-j4wc5\") pod \"ovnkube-node-wql74\" (UID: \"b7c67823-8460-44f5-a9e1-7aa98e7c7165\") " pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.277403 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.277378    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c44tw\" (UniqueName: \"kubernetes.io/projected/5efcbf66-b1f2-4c8a-a3af-54cba8053ef0-kube-api-access-c44tw\") pod \"aws-ebs-csi-driver-node-46cb8\" (UID: \"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.370025 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.369984    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/886ede06-fccf-4379-831a-41c244ab7387-dbus\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.370025 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370029    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/886ede06-fccf-4379-831a-41c244ab7387-kubelet-config\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370055    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370083    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-modprobe-d\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370126    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/886ede06-fccf-4379-831a-41c244ab7387-dbus\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-lib-modules\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/886ede06-fccf-4379-831a-41c244ab7387-kubelet-config\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-host\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370264 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.370263    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx26l\" (UniqueName: \"kubernetes.io/projected/5cba6262-d5b6-4e09-8b0f-78253daba24d-kube-api-access-wx26l\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370278    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-modprobe-d\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysctl-d\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-lib-modules\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-host\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.370355    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:20.870309859 +0000 UTC m=+3.131309579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxxsn\" (UniqueName: \"kubernetes.io/projected/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-kube-api-access-pxxsn\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370400    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysctl-d\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a91e5420-94f1-4941-82f5-1cf1eae0496c-agent-certs\") pod \"konnectivity-agent-mfvsh\" (UID: \"a91e5420-94f1-4941-82f5-1cf1eae0496c\") " pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370452    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkkqf\" (UniqueName: \"kubernetes.io/projected/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-kube-api-access-lkkqf\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-kubernetes\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysctl-conf\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-systemd\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5cba6262-d5b6-4e09-8b0f-78253daba24d-iptables-alerter-script\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-kubernetes\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370617    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-systemd\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.370727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cba6262-d5b6-4e09-8b0f-78253daba24d-host-slash\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370681    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-run\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysctl-conf\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-var-lib-kubelet\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-var-lib-kubelet\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370705    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cba6262-d5b6-4e09-8b0f-78253daba24d-host-slash\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370767    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370755    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-run\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370807    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhvsv\" (UniqueName: \"kubernetes.io/projected/f6571af9-eab4-4e45-8560-9c0065dcabfd-kube-api-access-hhvsv\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-host\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.370852    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370861    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-serviceca\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysconfig\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.370908    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:51:20.87089366 +0000 UTC m=+3.131893389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370919    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-host\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-sys\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370940    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-sysconfig\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370966    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-tuned\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.371630 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.370991    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-tmp\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.372581 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.371008    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-sys\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.372581 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.371020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a91e5420-94f1-4941-82f5-1cf1eae0496c-konnectivity-ca\") pod \"konnectivity-agent-mfvsh\" (UID: \"a91e5420-94f1-4941-82f5-1cf1eae0496c\") " pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.372581 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.371221    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5cba6262-d5b6-4e09-8b0f-78253daba24d-iptables-alerter-script\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.372581 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.371340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-serviceca\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.372581 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.371687    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a91e5420-94f1-4941-82f5-1cf1eae0496c-konnectivity-ca\") pod \"konnectivity-agent-mfvsh\" (UID: \"a91e5420-94f1-4941-82f5-1cf1eae0496c\") " pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.373227 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.373182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a91e5420-94f1-4941-82f5-1cf1eae0496c-agent-certs\") pod \"konnectivity-agent-mfvsh\" (UID: \"a91e5420-94f1-4941-82f5-1cf1eae0496c\") " pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.373517 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.373499    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-tmp\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.373611 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.373564    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-etc-tuned\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.380846 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.380783    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:20.380846 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.380814    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:20.380846 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.380831    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:20.381141 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.380886    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:20.880869381 +0000 UTC m=+3.141869093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:20.382587 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.382554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx26l\" (UniqueName: \"kubernetes.io/projected/5cba6262-d5b6-4e09-8b0f-78253daba24d-kube-api-access-wx26l\") pod \"iptables-alerter-rk7ps\" (UID: \"5cba6262-d5b6-4e09-8b0f-78253daba24d\") " pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.383010 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.382983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxxsn\" (UniqueName: \"kubernetes.io/projected/9e92f2d6-b9f9-44a3-a65d-b027a070bd84-kube-api-access-pxxsn\") pod \"tuned-jlr7n\" (UID: \"9e92f2d6-b9f9-44a3-a65d-b027a070bd84\") " pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.383904 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.383874    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhvsv\" (UniqueName: \"kubernetes.io/projected/f6571af9-eab4-4e45-8560-9c0065dcabfd-kube-api-access-hhvsv\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.384035 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.384016    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkkqf\" (UniqueName: \"kubernetes.io/projected/b2d08fc9-7fee-4c81-9b23-ffad61e60a7b-kube-api-access-lkkqf\") pod \"node-ca-nz8gs\" (UID: \"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b\") " pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.455986 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.455940    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8"
Jul 23 21:51:20.467423 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.467387    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vc8qb"
Jul 23 21:51:20.477752 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.477722    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8bn87"
Jul 23 21:51:20.482958 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.482933    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:20.492064 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.492036    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sftqj"
Jul 23 21:51:20.498950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.498923    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jlr7n"
Jul 23 21:51:20.502396 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.502363    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 21:51:20.507540 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.507520    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nz8gs"
Jul 23 21:51:20.516485 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.516458    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rk7ps"
Jul 23 21:51:20.523443 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.523423    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:20.875688 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.875622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:20.875863 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.875688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:20.875863 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.875779    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:20.875983 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.875860    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:20.875983 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.875884    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:21.875826638 +0000 UTC m=+4.136826351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:20.875983 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.875925    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:51:21.875906245 +0000 UTC m=+4.136905968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:20.976548 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:20.976508    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:20.976752 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.976724    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:20.976752 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.976752    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:20.976845 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.976766    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:20.976897 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:20.976833    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:21.976814273 +0000 UTC m=+4.237814001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:21.205730 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:21.205581    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 21:46:20 +0000 UTC" deadline="2028-05-04 05:31:41.930208556 +0000 UTC"
Jul 23 21:51:21.205730 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:21.205673    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15607h40m20.724540305s"
Jul 23 21:51:21.267550 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.267494    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b50422_b290_4646_babf_e85f78ea429c.slice/crio-da6d13a44c16c96ef40725429a80f49fe194fa5df6a81113c574bcd5933fa67b WatchSource:0}: Error finding container da6d13a44c16c96ef40725429a80f49fe194fa5df6a81113c574bcd5933fa67b: Status 404 returned error can't find the container with id da6d13a44c16c96ef40725429a80f49fe194fa5df6a81113c574bcd5933fa67b
Jul 23 21:51:21.269672 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.269611    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e92f2d6_b9f9_44a3_a65d_b027a070bd84.slice/crio-26009863f4d8f5b53d3d540a7c8ad62ef979c7eda4d72c45de89c36655d663e9 WatchSource:0}: Error finding container 26009863f4d8f5b53d3d540a7c8ad62ef979c7eda4d72c45de89c36655d663e9: Status 404 returned error can't find the container with id 26009863f4d8f5b53d3d540a7c8ad62ef979c7eda4d72c45de89c36655d663e9
Jul 23 21:51:21.270450 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.270425    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cba6262_d5b6_4e09_8b0f_78253daba24d.slice/crio-27d3f4e89c13db6b994704a86d5e8751a3ab6e76088e927a65dde6ecf4c3b0e2 WatchSource:0}: Error finding container 27d3f4e89c13db6b994704a86d5e8751a3ab6e76088e927a65dde6ecf4c3b0e2: Status 404 returned error can't find the container with id 27d3f4e89c13db6b994704a86d5e8751a3ab6e76088e927a65dde6ecf4c3b0e2
Jul 23 21:51:21.294569 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.293914    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efcbf66_b1f2_4c8a_a3af_54cba8053ef0.slice/crio-fe3bf1093390080e18e18f8ed78c9725cf5b165a75acaba2813530ed7d57c362 WatchSource:0}: Error finding container fe3bf1093390080e18e18f8ed78c9725cf5b165a75acaba2813530ed7d57c362: Status 404 returned error can't find the container with id fe3bf1093390080e18e18f8ed78c9725cf5b165a75acaba2813530ed7d57c362
Jul 23 21:51:21.298191 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.298100    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b2d5df_b34f_4985_b490_555ffb020fcc.slice/crio-4147f3d1df581fef3e672c9fe2394f9803f1862d9bdfc63e12a98567fc8716d3 WatchSource:0}: Error finding container 4147f3d1df581fef3e672c9fe2394f9803f1862d9bdfc63e12a98567fc8716d3: Status 404 returned error can't find the container with id 4147f3d1df581fef3e672c9fe2394f9803f1862d9bdfc63e12a98567fc8716d3
Jul 23 21:51:21.298885 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.298865    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5209be5a_03c5_4cf0_b6bc_dbf4acf85df3.slice/crio-f073c2d69befeef46f9a4571b14824ffffd49ced43f7f9667e53787267113de9 WatchSource:0}: Error finding container f073c2d69befeef46f9a4571b14824ffffd49ced43f7f9667e53787267113de9: Status 404 returned error can't find the container with id f073c2d69befeef46f9a4571b14824ffffd49ced43f7f9667e53787267113de9
Jul 23 21:51:21.300467 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:21.300448    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d08fc9_7fee_4c81_9b23_ffad61e60a7b.slice/crio-eb7caabeaa99f073ae6a89f259409bbafedb1a13e2034320051207e8ce99804d WatchSource:0}: Error finding container eb7caabeaa99f073ae6a89f259409bbafedb1a13e2034320051207e8ce99804d: Status 404 returned error can't find the container with id eb7caabeaa99f073ae6a89f259409bbafedb1a13e2034320051207e8ce99804d
Jul 23 21:51:21.883897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:21.883553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:21.884068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:21.883917    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:21.884136 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.883794    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:21.884204 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.884139    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:23.88411683 +0000 UTC m=+6.145116544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:21.884488 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.884067    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:21.884588 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.884526    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:51:23.884508304 +0000 UTC m=+6.145508031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:21.984358 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:21.984315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:21.984566 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.984508    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:21.984566 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.984531    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:21.984566 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.984546    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:21.984851 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:21.984607    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:23.984588145 +0000 UTC m=+6.245587866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:22.205996 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.205866    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 21:46:20 +0000 UTC" deadline="2028-02-19 05:04:33.819823856 +0000 UTC"
Jul 23 21:51:22.205996 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.205908    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13807h13m11.613920514s"
Jul 23 21:51:22.224239 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.222723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:22.224239 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:22.222870    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:22.224239 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.223322    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:22.224239 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:22.223437    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:22.224239 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.223530    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:22.224239 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:22.223636    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:22.259393 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.259217    2575 generic.go:356] "Generic (PLEG): container finished" podID="e82e16c6891169bbedf95a1d89739584" containerID="3bbfd2eaab33adee976ea9999e286a794683104e8bf266ba46c1738a9d1d097b" exitCode=0
Jul 23 21:51:22.259393 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.259328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal" event={"ID":"e82e16c6891169bbedf95a1d89739584","Type":"ContainerDied","Data":"3bbfd2eaab33adee976ea9999e286a794683104e8bf266ba46c1738a9d1d097b"}
Jul 23 21:51:22.280962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.280862    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal" event={"ID":"aa48ab49f2c0e7a214bbd553b8e1864c","Type":"ContainerStarted","Data":"4167313c1d88b0bd61362e273aa9a11d21be943b7642384c6a3d44335f2f6327"}
Jul 23 21:51:22.299473 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.299394    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-223.ec2.internal" podStartSLOduration=2.299369124 podStartE2EDuration="2.299369124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:22.298826492 +0000 UTC m=+4.559826224" watchObservedRunningTime="2026-07-23 21:51:22.299369124 +0000 UTC m=+4.560368858"
Jul 23 21:51:22.310242 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.310202    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"fbaf05a27aebed9d2818a86108fdf8110c4bb67ad0fc5df16b4a8c60ea86fd15"}
Jul 23 21:51:22.332470 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.332432    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mfvsh" event={"ID":"a91e5420-94f1-4941-82f5-1cf1eae0496c","Type":"ContainerStarted","Data":"3d44f529cda7ad3dfaca3575eee53bf52d1091adee6871b7d90b4056b8891d5a"}
Jul 23 21:51:22.334568 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.334537    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerStarted","Data":"f073c2d69befeef46f9a4571b14824ffffd49ced43f7f9667e53787267113de9"}
Jul 23 21:51:22.346527 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.346493    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sftqj" event={"ID":"d8b2d5df-b34f-4985-b490-555ffb020fcc","Type":"ContainerStarted","Data":"4147f3d1df581fef3e672c9fe2394f9803f1862d9bdfc63e12a98567fc8716d3"}
Jul 23 21:51:22.349804 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.349774    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rk7ps" event={"ID":"5cba6262-d5b6-4e09-8b0f-78253daba24d","Type":"ContainerStarted","Data":"27d3f4e89c13db6b994704a86d5e8751a3ab6e76088e927a65dde6ecf4c3b0e2"}
Jul 23 21:51:22.367814 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.367778    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jlr7n" event={"ID":"9e92f2d6-b9f9-44a3-a65d-b027a070bd84","Type":"ContainerStarted","Data":"26009863f4d8f5b53d3d540a7c8ad62ef979c7eda4d72c45de89c36655d663e9"}
Jul 23 21:51:22.386966 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.386926    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nz8gs" event={"ID":"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b","Type":"ContainerStarted","Data":"eb7caabeaa99f073ae6a89f259409bbafedb1a13e2034320051207e8ce99804d"}
Jul 23 21:51:22.404504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.404462    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8" event={"ID":"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0","Type":"ContainerStarted","Data":"fe3bf1093390080e18e18f8ed78c9725cf5b165a75acaba2813530ed7d57c362"}
Jul 23 21:51:22.421430 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:22.419413    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8bn87" event={"ID":"49b50422-b290-4646-babf-e85f78ea429c","Type":"ContainerStarted","Data":"da6d13a44c16c96ef40725429a80f49fe194fa5df6a81113c574bcd5933fa67b"}
Jul 23 21:51:23.461774 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:23.461628    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal" event={"ID":"e82e16c6891169bbedf95a1d89739584","Type":"ContainerStarted","Data":"cba167a29f13ab13d4ba5bbb73b1e46ca60f442aee8cf34390dd4c96eb6bd1a3"}
Jul 23 21:51:23.482419 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:23.482353    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-223.ec2.internal" podStartSLOduration=3.482329125 podStartE2EDuration="3.482329125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:23.481754411 +0000 UTC m=+5.742754145" watchObservedRunningTime="2026-07-23 21:51:23.482329125 +0000 UTC m=+5.743328858"
Jul 23 21:51:23.899234 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:23.899131    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:23.899234 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:23.899229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:23.899491 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:23.899373    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:23.899491 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:23.899439    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:51:27.899418258 +0000 UTC m=+10.160417983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:23.899620 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:23.899507    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:23.899620 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:23.899541    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:27.899529131 +0000 UTC m=+10.160528856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:24.000372 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:24.000330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:24.000625 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.000601    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:24.000734 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.000627    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:24.000734 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.000642    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:24.000848 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.000746    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:28.000725126 +0000 UTC m=+10.261724849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:24.221350 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:24.220796    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:24.221350 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:24.220804    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:24.221350 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.220937    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:24.221350 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.221045    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:24.221350 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:24.221210    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:24.221350 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:24.221307    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:26.221039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:26.220526    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:26.221039 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:26.220651    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:26.221039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:26.220759    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:26.221039 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:26.220882    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:26.221039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:26.220938    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:26.221039 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:26.221029    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:27.933603 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:27.933559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:27.934012 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:27.933672    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:27.934012 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:27.933738    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:27.934012 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:27.933783    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:27.934012 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:27.933811    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:51:35.933788907 +0000 UTC m=+18.194788616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:27.934012 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:27.933890    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:35.933873879 +0000 UTC m=+18.194873590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:28.034438 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:28.034398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:28.034663 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.034643    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:28.034743 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.034669    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:28.034743 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.034683    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:28.034846 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.034744    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:36.034725006 +0000 UTC m=+18.295724727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:28.222130 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:28.221314    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:28.222130 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.221422    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:28.222130 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:28.221812    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:28.222130 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.221918    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:28.222130 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:28.221974    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:28.222130 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:28.222045    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:30.221450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:30.220720    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:30.221450 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:30.220847    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:30.221450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:30.221210    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:30.221450 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:30.221288    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:30.221450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:30.221336    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:30.221450 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:30.221413    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:32.221217 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:32.221130    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:32.221217 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:32.221172    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:32.221674 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:32.221143    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:32.221674 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:32.221270    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:32.221674 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:32.221337    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:32.221674 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:32.221430    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:34.220551 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:34.220514    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:34.221036 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:34.220658    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:34.221036 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:34.220659    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:34.221036 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:34.220734    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:34.221036 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:34.220833    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:34.221036 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:34.220928    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:35.995073 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:35.995040    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:35.995545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:35.995109    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:35.995545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:35.995189    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:35.995545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:35.995264    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:51.995243132 +0000 UTC m=+34.256242855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:35.995545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:35.995192    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:35.995545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:35.995347    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:51:51.995331432 +0000 UTC m=+34.256331143 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:36.095731 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:36.095657    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:36.095891 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.095827    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:36.095891 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.095849    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:36.095891 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.095862    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:36.096018 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.095914    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:52.095901031 +0000 UTC m=+34.356900741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:36.220648 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:36.220593    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:36.220820 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:36.220600    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:36.220820 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.220736    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:36.220820 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:36.220757    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:36.220971 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.220848    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:36.220971 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:36.220926    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:38.221829 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:38.221794    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:38.222279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:38.221901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:38.222279 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:38.221942    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:38.222279 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:38.222007    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:38.222279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:38.222042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:38.222279 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:38.222128    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:40.223343 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.223016    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:40.223343 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:40.223138    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:40.223777 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.223442    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:40.223777 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.223466    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:40.223777 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:40.223597    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:40.223777 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:40.223721    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:40.497616 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.497274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mfvsh" event={"ID":"a91e5420-94f1-4941-82f5-1cf1eae0496c","Type":"ContainerStarted","Data":"51160a9fe9387784d41678c2c38ab51baa54a0dacfd7ab6d07d29126feee5582"}
Jul 23 21:51:40.499012 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.498978    2575 generic.go:356] "Generic (PLEG): container finished" podID="5209be5a-03c5-4cf0-b6bc-dbf4acf85df3" containerID="97be6b0a8f03cf8b5844c1887f304ba6a969847802b50f17c4e5877799c2a12c" exitCode=0
Jul 23 21:51:40.499145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.499078    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerDied","Data":"97be6b0a8f03cf8b5844c1887f304ba6a969847802b50f17c4e5877799c2a12c"}
Jul 23 21:51:40.502276 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.502247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sftqj" event={"ID":"d8b2d5df-b34f-4985-b490-555ffb020fcc","Type":"ContainerStarted","Data":"0e19d98b96f3898479e648d86f029de270086341e8537aecd793b6fa3c881a48"}
Jul 23 21:51:40.504181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.504158    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jlr7n" event={"ID":"9e92f2d6-b9f9-44a3-a65d-b027a070bd84","Type":"ContainerStarted","Data":"18314b3230dc0b9d2dadc39afde210b9a5856e40d2996354cf1437da687b0d22"}
Jul 23 21:51:40.505625 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.505600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nz8gs" event={"ID":"b2d08fc9-7fee-4c81-9b23-ffad61e60a7b","Type":"ContainerStarted","Data":"476ae71810381aa9ff33f7b8bf10ce0cf99c3c8b8aa4f3c6609f9738a9a0f212"}
Jul 23 21:51:40.507224 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.507201    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8" event={"ID":"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0","Type":"ContainerStarted","Data":"41ba0aad9cc635bc64ccd6abd153214a5e1c4d730298a34bc38e3516d7bd493e"}
Jul 23 21:51:40.508617 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.508594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8bn87" event={"ID":"49b50422-b290-4646-babf-e85f78ea429c","Type":"ContainerStarted","Data":"417324bcafd4c3ae59900341a11065ac1d120fc11b529027610361538d4f340a"}
Jul 23 21:51:40.510235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.510217    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"4b352da6b110581fb8781db73e850edb6370d47e843bf9aa33ea746d7d9a6196"}
Jul 23 21:51:40.510315 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.510266    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"a9b8603fb1209e3ab73b34bf00051c6670fd25dea865dae00d87c117124c1290"}
Jul 23 21:51:40.514779 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.514733    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mfvsh" podStartSLOduration=7.07332619 podStartE2EDuration="21.514716691s" totalImagesPullingTime="14.441390501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:40.514203607 +0000 UTC m=+22.775203335" watchObservedRunningTime="2026-07-23 21:51:40.514716691 +0000 UTC m=+22.775716420"
Jul 23 21:51:40.528534 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.528293    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sftqj" podStartSLOduration=2.676716991 podStartE2EDuration="21.528280773s" totalImagesPullingTime="18.851563782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:40.528112738 +0000 UTC m=+22.789112481" watchObservedRunningTime="2026-07-23 21:51:40.528280773 +0000 UTC m=+22.789280501"
Jul 23 21:51:40.545282 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.545243    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jlr7n" podStartSLOduration=2.680617217 podStartE2EDuration="21.545229873s" totalImagesPullingTime="18.864612656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:40.544966954 +0000 UTC m=+22.805966703" watchObservedRunningTime="2026-07-23 21:51:40.545229873 +0000 UTC m=+22.806229601"
Jul 23 21:51:40.558934 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.558895    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nz8gs" podStartSLOduration=2.707948537 podStartE2EDuration="21.558881376s" totalImagesPullingTime="18.850932839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:40.558713793 +0000 UTC m=+22.819713540" watchObservedRunningTime="2026-07-23 21:51:40.558881376 +0000 UTC m=+22.819881104"
Jul 23 21:51:40.633184 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:40.633139    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8bn87" podStartSLOduration=3.718818598 podStartE2EDuration="22.633124376s" totalImagesPullingTime="18.914305778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:40.596657648 +0000 UTC m=+22.857657393" watchObservedRunningTime="2026-07-23 21:51:40.633124376 +0000 UTC m=+22.894124104"
Jul 23 21:51:41.515710 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:41.515676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"ba69cd235a267b255e543d508612976ad49dd42108c6fb0ded96dc13693ace70"}
Jul 23 21:51:41.516398 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:41.515801    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"e650295e3650a9f37912b50f1e8f9d504b587655650163a82f2d83e23aa8df41"}
Jul 23 21:51:41.516398 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:41.515852    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"348f34c793b8477feaaeb02933f6ad168f67860bf1e8351a1f41b2b15c2464fa"}
Jul 23 21:51:41.516398 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:41.515885    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"c841b2829f217e3b69dd834a18b37effcf365c3fdbbfd1df1bf170f36b93cd2f"}
Jul 23 21:51:41.543796 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:41.543773    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 21:51:42.219781 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.219686    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T21:51:41.5437935Z","UUID":"77766a8c-3b66-45c6-9185-f99595953c5f","Handler":null,"Name":"","Endpoint":""}
Jul 23 21:51:42.220493 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.220463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:42.220611 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.220463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:42.220611 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:42.220584    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:42.220748 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.220478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:42.220804 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:42.220747    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:42.220860 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:42.220835    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:42.221299 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.221281    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 21:51:42.221437 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.221317    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 21:51:42.520608 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.520525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rk7ps" event={"ID":"5cba6262-d5b6-4e09-8b0f-78253daba24d","Type":"ContainerStarted","Data":"8f1a937cff8b10f3fe07e948a49c947d51d4dc27ae48c3c12485614d52a97420"}
Jul 23 21:51:42.523942 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.523315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8" event={"ID":"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0","Type":"ContainerStarted","Data":"f3452d8d303bbe0988310313d64991a970afba739a74694866cd1ec24cd67a60"}
Jul 23 21:51:42.536436 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:42.536377    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rk7ps" podStartSLOduration=4.676672977 podStartE2EDuration="23.536356823s" totalImagesPullingTime="18.859683846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:42.535544131 +0000 UTC m=+24.796543884" watchObservedRunningTime="2026-07-23 21:51:42.536356823 +0000 UTC m=+24.797356552"
Jul 23 21:51:43.529124 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:43.528889    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"db4127597a322e0273b96652c8c5b86793b75506d6be865d7c7c1a7b3c12bf1b"}
Jul 23 21:51:43.531171 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:43.531144    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8" event={"ID":"5efcbf66-b1f2-4c8a-a3af-54cba8053ef0","Type":"ContainerStarted","Data":"2b19921aa566843089301007c4da678f33d05c64e17e10e255dcd6c6c92e5ec3"}
Jul 23 21:51:43.549728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:43.549675    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-46cb8" podStartSLOduration=4.335221234 podStartE2EDuration="25.54965715s" totalImagesPullingTime="21.214435916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:43.549411536 +0000 UTC m=+25.810411261" watchObservedRunningTime="2026-07-23 21:51:43.54965715 +0000 UTC m=+25.810656874"
Jul 23 21:51:44.002311 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.002230    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:44.002933 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.002910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:44.220711 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.220675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:44.220711 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.220707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:44.220930 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.220676    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:44.220930 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:44.220818    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:44.220930 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:44.220905    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:44.221086 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:44.220989    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:44.533504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.533461    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:44.534058 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:44.534005    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mfvsh"
Jul 23 21:51:45.539554 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:45.539144    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" event={"ID":"b7c67823-8460-44f5-a9e1-7aa98e7c7165","Type":"ContainerStarted","Data":"324eff87abed09d0817bef39b1a1eeb1ce3b988878df13da5827fba2e4d3cf48"}
Jul 23 21:51:45.539554 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:45.539468    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:45.545163 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:45.545118    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerDied","Data":"19a57c7ff533f221d560f512dab53c7b6924e57e48581fde47603f1aa80549c0"}
Jul 23 21:51:45.545518 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:45.545458    2575 generic.go:356] "Generic (PLEG): container finished" podID="5209be5a-03c5-4cf0-b6bc-dbf4acf85df3" containerID="19a57c7ff533f221d560f512dab53c7b6924e57e48581fde47603f1aa80549c0" exitCode=0
Jul 23 21:51:45.556233 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:45.556210    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:45.567871 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:45.567825    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wql74" podStartSLOduration=7.642875045 podStartE2EDuration="26.567811056s" totalImagesPullingTime="18.924936011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:45.566678426 +0000 UTC m=+27.827678173" watchObservedRunningTime="2026-07-23 21:51:45.567811056 +0000 UTC m=+27.828810814"
Jul 23 21:51:46.220521 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:46.220490    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:46.220785 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:46.220491    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:46.220785 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:46.220604    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:46.220785 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:46.220610    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:46.220785 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:46.220731    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:46.220980 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:46.220819    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:46.548142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:46.548062    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:46.548609 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:46.548548    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:46.562774 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:46.562741    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:51:47.257994 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.257956    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-29qg9"]
Jul 23 21:51:47.258211 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.258127    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:47.258321 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:47.258296    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:47.260847 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.260823    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tq988"]
Jul 23 21:51:47.260988 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.260926    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:47.261040 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:47.261007    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:47.263543 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.263522    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9t6qd"]
Jul 23 21:51:47.263683 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.263626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:47.263771 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:47.263751    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:47.552377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.552286    2575 generic.go:356] "Generic (PLEG): container finished" podID="5209be5a-03c5-4cf0-b6bc-dbf4acf85df3" containerID="0014af794ff4dca93fb675e0ce29b7718c2cef38f2d936c504608e434d5a21ab" exitCode=0
Jul 23 21:51:47.552377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:47.552348    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerDied","Data":"0014af794ff4dca93fb675e0ce29b7718c2cef38f2d936c504608e434d5a21ab"}
Jul 23 21:51:49.221281 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:49.221091    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:49.221668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:49.221109    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:49.221668 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:49.221354    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:49.221668 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:49.221442    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:49.221668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:49.221146    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:49.221668 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:49.221520    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:49.558711 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:49.558622    2575 generic.go:356] "Generic (PLEG): container finished" podID="5209be5a-03c5-4cf0-b6bc-dbf4acf85df3" containerID="71e2607c430634635084f9da456cee3b3c249147c36fe6925cfebc530f3a0c87" exitCode=0
Jul 23 21:51:49.558711 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:49.558669    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerDied","Data":"71e2607c430634635084f9da456cee3b3c249147c36fe6925cfebc530f3a0c87"}
Jul 23 21:51:51.220335 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:51.220305    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:51.221039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:51.220339    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:51.221039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:51.220305    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:51.221039 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:51.220431    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-29qg9" podUID="886ede06-fccf-4379-831a-41c244ab7387"
Jul 23 21:51:51.221039 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:51.220521    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9t6qd" podUID="f6571af9-eab4-4e45-8560-9c0065dcabfd"
Jul 23 21:51:51.221039 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:51.220587    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tq988" podUID="2a3d3f22-dfbf-4d92-8704-0b27641d38a8"
Jul 23 21:51:52.016701 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:52.016649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:52.016889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:52.016726    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:52.016889 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.016824    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:52.016889 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.016882    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:52:24.016868007 +0000 UTC m=+66.277867717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 21:51:52.017060 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.016824    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:52.017060 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.016923    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret podName:886ede06-fccf-4379-831a-41c244ab7387 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:24.016914875 +0000 UTC m=+66.277914594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret") pod "global-pull-secret-syncer-29qg9" (UID: "886ede06-fccf-4379-831a-41c244ab7387") : object "kube-system"/"original-pull-secret" not registered
Jul 23 21:51:52.117752 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:52.117715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:52.117906 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.117866    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 21:51:52.117906 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.117882    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 21:51:52.117906 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.117892    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7sbds for pod openshift-network-diagnostics/network-check-target-tq988: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:52.118019 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:52.117937    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds podName:2a3d3f22-dfbf-4d92-8704-0b27641d38a8 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:24.117925534 +0000 UTC m=+66.378925246 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7sbds" (UniqueName: "kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds") pod "network-check-target-tq988" (UID: "2a3d3f22-dfbf-4d92-8704-0b27641d38a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 21:51:53.049402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.049325    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-223.ec2.internal" event="NodeReady"
Jul 23 21:51:53.049899 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.049473    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 21:51:53.086610 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.086571    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74f864dbdd-2s7xr"]
Jul 23 21:51:53.112344 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.112310    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74f864dbdd-2s7xr"]
Jul 23 21:51:53.112502 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.112352    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v6d4c"]
Jul 23 21:51:53.112567 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.112511    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.115960 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.115931    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 21:51:53.116105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.115998    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rdwr9\""
Jul 23 21:51:53.116105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.116087    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 21:51:53.116105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.116089    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 21:51:53.122737 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.122712    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 21:51:53.124519 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.124499    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hjj5g"]
Jul 23 21:51:53.124663 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.124617    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.128170 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.128149    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 21:51:53.128298 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.128151    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 21:51:53.128298 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.128189    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b7bnz\""
Jul 23 21:51:53.142440 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.142417    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v6d4c"]
Jul 23 21:51:53.142565 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.142460    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hjj5g"]
Jul 23 21:51:53.142565 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.142552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.145064 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.145037    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 21:51:53.145173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.145099    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qfzdd\""
Jul 23 21:51:53.145173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.145138    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 21:51:53.145173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.145145    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 21:51:53.221108 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.221073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:51:53.221277 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.221187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:51:53.221277 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.221217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:51:53.224365 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.224342    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jd2p5\""
Jul 23 21:51:53.224647 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.224614    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 21:51:53.224849 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.224834    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 21:51:53.225092 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.225072    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 21:51:53.225214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.225196    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 21:51:53.225480 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.225461    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sc2x2\""
Jul 23 21:51:53.227022 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227000    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-installation-pull-secrets\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227132 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227057    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kswpd\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-kube-api-access-kswpd\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227132 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.227132 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-image-registry-private-configuration\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227291 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmxz\" (UniqueName: \"kubernetes.io/projected/27d52fef-85f6-496c-b787-1f334f4c3f33-kube-api-access-sxmxz\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.227291 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddf53267-738e-4f2f-852b-9ea39603709c-cert\") pod \"ingress-canary-hjj5g\" (UID: \"ddf53267-738e-4f2f-852b-9ea39603709c\") " pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.227291 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27d52fef-85f6-496c-b787-1f334f4c3f33-tmp-dir\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.227390 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txhd8\" (UniqueName: \"kubernetes.io/projected/ddf53267-738e-4f2f-852b-9ea39603709c-kube-api-access-txhd8\") pod \"ingress-canary-hjj5g\" (UID: \"ddf53267-738e-4f2f-852b-9ea39603709c\") " pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.227430 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34152caf-86bd-4ff2-8f02-7ea46621d1af-ca-trust-extracted\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227430 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227420    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27d52fef-85f6-496c-b787-1f334f4c3f33-config-volume\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.227506 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-trusted-ca\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227506 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-bound-sa-token\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227599 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227523    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-certificates\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.227599 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.227551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.286361 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.286325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:53.328287 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-bound-sa-token\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.328287 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-certificates\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.328287 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328279    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-installation-pull-secrets\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kswpd\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-kube-api-access-kswpd\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-image-registry-private-configuration\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328453    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmxz\" (UniqueName: \"kubernetes.io/projected/27d52fef-85f6-496c-b787-1f334f4c3f33-kube-api-access-sxmxz\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddf53267-738e-4f2f-852b-9ea39603709c-cert\") pod \"ingress-canary-hjj5g\" (UID: \"ddf53267-738e-4f2f-852b-9ea39603709c\") " pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.328486    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27d52fef-85f6-496c-b787-1f334f4c3f33-tmp-dir\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.328510    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f864dbdd-2s7xr: secret "image-registry-tls" not found
Jul 23 21:51:53.328545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328526    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txhd8\" (UniqueName: \"kubernetes.io/projected/ddf53267-738e-4f2f-852b-9ea39603709c-kube-api-access-txhd8\") pod \"ingress-canary-hjj5g\" (UID: \"ddf53267-738e-4f2f-852b-9ea39603709c\") " pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.329099 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.328584    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls podName:34152caf-86bd-4ff2-8f02-7ea46621d1af nodeName:}" failed. No retries permitted until 2026-07-23 21:51:53.828563276 +0000 UTC m=+36.089563001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls") pod "image-registry-74f864dbdd-2s7xr" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af") : secret "image-registry-tls" not found
Jul 23 21:51:53.329099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328562    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34152caf-86bd-4ff2-8f02-7ea46621d1af-ca-trust-extracted\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.329099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27d52fef-85f6-496c-b787-1f334f4c3f33-config-volume\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.329099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.328661    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-trusted-ca\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.329099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.329014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-certificates\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.329349 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.329225    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34152caf-86bd-4ff2-8f02-7ea46621d1af-ca-trust-extracted\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.329349 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.329249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27d52fef-85f6-496c-b787-1f334f4c3f33-tmp-dir\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.329452 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.329364    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 21:51:53.329452 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.329432    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls podName:27d52fef-85f6-496c-b787-1f334f4c3f33 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:53.829411336 +0000 UTC m=+36.090411051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls") pod "dns-default-v6d4c" (UID: "27d52fef-85f6-496c-b787-1f334f4c3f33") : secret "dns-default-metrics-tls" not found
Jul 23 21:51:53.329544 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.329486    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27d52fef-85f6-496c-b787-1f334f4c3f33-config-volume\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.329601 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.329578    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-trusted-ca\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.333056 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.333030    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-installation-pull-secrets\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.333201 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.333113    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-image-registry-private-configuration\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.333262 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.333216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddf53267-738e-4f2f-852b-9ea39603709c-cert\") pod \"ingress-canary-hjj5g\" (UID: \"ddf53267-738e-4f2f-852b-9ea39603709c\") " pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.345156 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.345134    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-bound-sa-token\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.359344 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.359312    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kswpd\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-kube-api-access-kswpd\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.359854 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.359831    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txhd8\" (UniqueName: \"kubernetes.io/projected/ddf53267-738e-4f2f-852b-9ea39603709c-kube-api-access-txhd8\") pod \"ingress-canary-hjj5g\" (UID: \"ddf53267-738e-4f2f-852b-9ea39603709c\") " pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.366794 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.366769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmxz\" (UniqueName: \"kubernetes.io/projected/27d52fef-85f6-496c-b787-1f334f4c3f33-kube-api-access-sxmxz\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.429668 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.429612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:53.453577 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.453545    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hjj5g"
Jul 23 21:51:53.831456 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.831203    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:53.831573 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.831350    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 21:51:53.831573 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.831524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:53.831656 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.831535    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f864dbdd-2s7xr: secret "image-registry-tls" not found
Jul 23 21:51:53.831713 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.831580    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 21:51:53.831743 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.831703    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls podName:34152caf-86bd-4ff2-8f02-7ea46621d1af nodeName:}" failed. No retries permitted until 2026-07-23 21:51:54.831679128 +0000 UTC m=+37.092678844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls") pod "image-registry-74f864dbdd-2s7xr" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af") : secret "image-registry-tls" not found
Jul 23 21:51:53.831780 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:53.831767    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls podName:27d52fef-85f6-496c-b787-1f334f4c3f33 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:54.831748003 +0000 UTC m=+37.092747714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls") pod "dns-default-v6d4c" (UID: "27d52fef-85f6-496c-b787-1f334f4c3f33") : secret "dns-default-metrics-tls" not found
Jul 23 21:51:53.894193 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:53.894162    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hjj5g"]
Jul 23 21:51:53.977017 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:51:53.976915    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf53267_738e_4f2f_852b_9ea39603709c.slice/crio-d2e53a6d7435d08b5a326bd2e37f48e2b5f275dc0f755153add3502a23754b02 WatchSource:0}: Error finding container d2e53a6d7435d08b5a326bd2e37f48e2b5f275dc0f755153add3502a23754b02: Status 404 returned error can't find the container with id d2e53a6d7435d08b5a326bd2e37f48e2b5f275dc0f755153add3502a23754b02
Jul 23 21:51:54.575804 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:54.575766    2575 generic.go:356] "Generic (PLEG): container finished" podID="5209be5a-03c5-4cf0-b6bc-dbf4acf85df3" containerID="1680e51659f5a25a811b5b8bfb6076e68611e5cece808cea9c37502181fb2700" exitCode=0
Jul 23 21:51:54.576415 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:54.575855    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerDied","Data":"1680e51659f5a25a811b5b8bfb6076e68611e5cece808cea9c37502181fb2700"}
Jul 23 21:51:54.577752 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:54.577727    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hjj5g" event={"ID":"ddf53267-738e-4f2f-852b-9ea39603709c","Type":"ContainerStarted","Data":"d2e53a6d7435d08b5a326bd2e37f48e2b5f275dc0f755153add3502a23754b02"}
Jul 23 21:51:54.838391 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:54.838307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:54.838541 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:54.838409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:54.838541 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:54.838466    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 21:51:54.838658 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:54.838545    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls podName:27d52fef-85f6-496c-b787-1f334f4c3f33 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:56.838523584 +0000 UTC m=+39.099523297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls") pod "dns-default-v6d4c" (UID: "27d52fef-85f6-496c-b787-1f334f4c3f33") : secret "dns-default-metrics-tls" not found
Jul 23 21:51:54.838658 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:54.838575    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 21:51:54.838658 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:54.838596    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f864dbdd-2s7xr: secret "image-registry-tls" not found
Jul 23 21:51:54.838812 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:54.838661    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls podName:34152caf-86bd-4ff2-8f02-7ea46621d1af nodeName:}" failed. No retries permitted until 2026-07-23 21:51:56.838645811 +0000 UTC m=+39.099645520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls") pod "image-registry-74f864dbdd-2s7xr" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af") : secret "image-registry-tls" not found
Jul 23 21:51:55.583326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:55.583290    2575 generic.go:356] "Generic (PLEG): container finished" podID="5209be5a-03c5-4cf0-b6bc-dbf4acf85df3" containerID="ba7ddfa90ec29a395405c929d17522bb86da3dda8857d308581812ebf2e2241f" exitCode=0
Jul 23 21:51:55.583793 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:55.583327    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerDied","Data":"ba7ddfa90ec29a395405c929d17522bb86da3dda8857d308581812ebf2e2241f"}
Jul 23 21:51:56.587827 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.587620    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hjj5g" event={"ID":"ddf53267-738e-4f2f-852b-9ea39603709c","Type":"ContainerStarted","Data":"748a2532cae9668dc877422775e4597a939592f2410a9095daaefcf695f2b846"}
Jul 23 21:51:56.590832 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.590804    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" event={"ID":"5209be5a-03c5-4cf0-b6bc-dbf4acf85df3","Type":"ContainerStarted","Data":"85775e94f549f7ed5f192bc906fde680e95721a290c722d2402345eb77546f96"}
Jul 23 21:51:56.604716 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.604673    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hjj5g" podStartSLOduration=1.279574272 podStartE2EDuration="3.604657522s" totalImagesPullingTime="2.32508325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:56.603969641 +0000 UTC m=+38.864969380" watchObservedRunningTime="2026-07-23 21:51:56.604657522 +0000 UTC m=+38.865657250"
Jul 23 21:51:56.625211 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.625152    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vc8qb" podStartSLOduration=8.55327854 podStartE2EDuration="38.625136176s" totalImagesPullingTime="30.071857636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:56.62408646 +0000 UTC m=+38.885086188" watchObservedRunningTime="2026-07-23 21:51:56.625136176 +0000 UTC m=+38.886135908"
Jul 23 21:51:56.750144 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.750110    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:56.815946 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.815908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:56.854730 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.854656    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:51:56.854730 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:56.854730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:51:56.854879 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:56.854789    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 21:51:56.854879 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:56.854810    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 21:51:56.854879 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:56.854820    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f864dbdd-2s7xr: secret "image-registry-tls" not found
Jul 23 21:51:56.854879 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:56.854848    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls podName:27d52fef-85f6-496c-b787-1f334f4c3f33 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:00.854831703 +0000 UTC m=+43.115831413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls") pod "dns-default-v6d4c" (UID: "27d52fef-85f6-496c-b787-1f334f4c3f33") : secret "dns-default-metrics-tls" not found
Jul 23 21:51:56.854879 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:51:56.854865    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls podName:34152caf-86bd-4ff2-8f02-7ea46621d1af nodeName:}" failed. No retries permitted until 2026-07-23 21:52:00.854854764 +0000 UTC m=+43.115854471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls") pod "image-registry-74f864dbdd-2s7xr" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af") : secret "image-registry-tls" not found
Jul 23 21:51:57.237659 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:57.237569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:57.338035 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:57.337998    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:58.898698 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:58.898662    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:58.992959 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:58.992925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:59.803749 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:59.803710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:51:59.904724 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:51:59.904693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:00.006957 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:00.006916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:00.070450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:00.070367    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:00.832883 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:00.832850    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:00.889912 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:00.889878    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:00.890064 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:00.889943    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:00.890064 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:00.890026    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 21:52:00.890064 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:00.890042    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f864dbdd-2s7xr: secret "image-registry-tls" not found
Jul 23 21:52:00.890064 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:00.890051    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 21:52:00.890191 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:00.890097    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls podName:34152caf-86bd-4ff2-8f02-7ea46621d1af nodeName:}" failed. No retries permitted until 2026-07-23 21:52:08.89008071 +0000 UTC m=+51.151080417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls") pod "image-registry-74f864dbdd-2s7xr" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af") : secret "image-registry-tls" not found
Jul 23 21:52:00.890191 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:00.890110    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls podName:27d52fef-85f6-496c-b787-1f334f4c3f33 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:08.890104593 +0000 UTC m=+51.151104299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls") pod "dns-default-v6d4c" (UID: "27d52fef-85f6-496c-b787-1f334f4c3f33") : secret "dns-default-metrics-tls" not found
Jul 23 21:52:01.099442 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.099402    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"]
Jul 23 21:52:01.131806 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.131773    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"]
Jul 23 21:52:01.131940 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.131889    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.134601 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.134579    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 21:52:01.135706 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.135692    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dfcgf\""
Jul 23 21:52:01.136455 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.136432    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 21:52:01.136539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.136501    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 23 21:52:01.136539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.136506    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 23 21:52:01.199348 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.199317    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"]
Jul 23 21:52:01.217592 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.217557    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:01.225506 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.225475    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"]
Jul 23 21:52:01.225656 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.225602    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:01.228264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.228222    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 23 21:52:01.228264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.228258    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pfn8f\""
Jul 23 21:52:01.228518 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.228503    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:52:01.229052 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.229037    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 23 21:52:01.249732 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.249710    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s9bnx"]
Jul 23 21:52:01.249852 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.249837    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"
Jul 23 21:52:01.254286 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.254262    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 23 21:52:01.254286 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.254279    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:52:01.254434 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.254323    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h2bzl\""
Jul 23 21:52:01.273720 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.273699    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"]
Jul 23 21:52:01.273837 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.273744    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"]
Jul 23 21:52:01.273837 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.273763    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s9bnx"]
Jul 23 21:52:01.273837 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.273818    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.276168 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.276149    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 23 21:52:01.276259 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.276183    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:52:01.276259 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.276192    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 23 21:52:01.276259 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.276149    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dpd7h\""
Jul 23 21:52:01.276605 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.276591    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 23 21:52:01.283698 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.283680    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 23 21:52:01.292440 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.292419    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rg5c\" (UniqueName: \"kubernetes.io/projected/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-kube-api-access-5rg5c\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.292530 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.292449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.292530 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.292474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.299119 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.299091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:01.304389 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.304369    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-4k9jn"]
Jul 23 21:52:01.323999 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.323976    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f9b7f69fb-bxs2f"]
Jul 23 21:52:01.324120 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.324051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.326041 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.326018    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 21:52:01.326291 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.326273    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 21:52:01.326425 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.326410    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 23 21:52:01.326500 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.326485    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 23 21:52:01.326937 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.326872    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7ww2w\""
Jul 23 21:52:01.332185 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.332164    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 23 21:52:01.350021 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.349969    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-4k9jn"]
Jul 23 21:52:01.350021 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.349999    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f9b7f69fb-bxs2f"]
Jul 23 21:52:01.350138 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.350077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.352266 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352251    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 23 21:52:01.352343 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352251    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 23 21:52:01.352406 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352381    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mqtwk\""
Jul 23 21:52:01.352406 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352390    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 23 21:52:01.352519 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352505    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 23 21:52:01.352731 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352718    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 21:52:01.352774 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.352744    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 21:52:01.393280 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rg5c\" (UniqueName: \"kubernetes.io/projected/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-kube-api-access-5rg5c\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.393457 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-566gs\" (UniqueName: \"kubernetes.io/projected/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-kube-api-access-566gs\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:01.393457 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.393457 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393331    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2spf6\" (UniqueName: \"kubernetes.io/projected/9d931f74-1cd1-40fb-ba18-c613477a3784-kube-api-access-2spf6\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.393457 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpdv\" (UniqueName: \"kubernetes.io/projected/09c67a01-6139-4677-b7b1-0cf92777ccfb-kube-api-access-fwpdv\") pod \"volume-data-source-validator-78cf4c6f6-67d5s\" (UID: \"09c67a01-6139-4677-b7b1-0cf92777ccfb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"
Jul 23 21:52:01.393457 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.393739 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.393475    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:01.393739 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.393565    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls podName:335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:01.893539867 +0000 UTC m=+44.154539593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n9zrt" (UID: "335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58") : secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:01.393739 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d931f74-1cd1-40fb-ba18-c613477a3784-trusted-ca\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.393739 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393708    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:01.393884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d931f74-1cd1-40fb-ba18-c613477a3784-serving-cert\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.393884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.393794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d931f74-1cd1-40fb-ba18-c613477a3784-config\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.394101 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.394085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.408722 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.408698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rg5c\" (UniqueName: \"kubernetes.io/projected/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-kube-api-access-5rg5c\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.468316 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.468278    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:01.494475 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ea1e6f27-49dc-457b-871f-699f016f9bdf-snapshots\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.494604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea1e6f27-49dc-457b-871f-699f016f9bdf-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.494604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d931f74-1cd1-40fb-ba18-c613477a3784-trusted-ca\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.494604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stdmh\" (UniqueName: \"kubernetes.io/projected/ea1e6f27-49dc-457b-871f-699f016f9bdf-kube-api-access-stdmh\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.494604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-566gs\" (UniqueName: \"kubernetes.io/projected/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-kube-api-access-566gs\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:01.494604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d931f74-1cd1-40fb-ba18-c613477a3784-config\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.494604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl7sx\" (UniqueName: \"kubernetes.io/projected/cdcef278-0391-4f9d-b80c-7f404fb55655-kube-api-access-nl7sx\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.494950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494738    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea1e6f27-49dc-457b-871f-699f016f9bdf-service-ca-bundle\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.494950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494849    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpdv\" (UniqueName: \"kubernetes.io/projected/09c67a01-6139-4677-b7b1-0cf92777ccfb-kube-api-access-fwpdv\") pod \"volume-data-source-validator-78cf4c6f6-67d5s\" (UID: \"09c67a01-6139-4677-b7b1-0cf92777ccfb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"
Jul 23 21:52:01.494950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-default-certificate\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.494950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494907    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea1e6f27-49dc-457b-871f-699f016f9bdf-tmp\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494957    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.494994    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d931f74-1cd1-40fb-ba18-c613477a3784-serving-cert\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-stats-auth\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.495084    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 21:52:01.495145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea1e6f27-49dc-457b-871f-699f016f9bdf-serving-cert\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.495464 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.495182    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls podName:79a5c457-e4c5-4fa6-930f-9b6ca5a8b262 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:01.995163808 +0000 UTC m=+44.256163513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fp62t" (UID: "79a5c457-e4c5-4fa6-930f-9b6ca5a8b262") : secret "samples-operator-tls" not found
Jul 23 21:52:01.495464 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2spf6\" (UniqueName: \"kubernetes.io/projected/9d931f74-1cd1-40fb-ba18-c613477a3784-kube-api-access-2spf6\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.495464 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d931f74-1cd1-40fb-ba18-c613477a3784-config\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.495687 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.495667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d931f74-1cd1-40fb-ba18-c613477a3784-trusted-ca\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.497327 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.497306    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d931f74-1cd1-40fb-ba18-c613477a3784-serving-cert\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.503134 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.503104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2spf6\" (UniqueName: \"kubernetes.io/projected/9d931f74-1cd1-40fb-ba18-c613477a3784-kube-api-access-2spf6\") pod \"console-operator-565b7b569d-s9bnx\" (UID: \"9d931f74-1cd1-40fb-ba18-c613477a3784\") " pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.504055 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.504025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-566gs\" (UniqueName: \"kubernetes.io/projected/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-kube-api-access-566gs\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:01.504232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.504214    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpdv\" (UniqueName: \"kubernetes.io/projected/09c67a01-6139-4677-b7b1-0cf92777ccfb-kube-api-access-fwpdv\") pod \"volume-data-source-validator-78cf4c6f6-67d5s\" (UID: \"09c67a01-6139-4677-b7b1-0cf92777ccfb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"
Jul 23 21:52:01.558479 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.558441    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"
Jul 23 21:52:01.570400 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.570356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:01.584210 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.584188    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:01.596524 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596503    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ea1e6f27-49dc-457b-871f-699f016f9bdf-snapshots\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.596603 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596534    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea1e6f27-49dc-457b-871f-699f016f9bdf-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.596603 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596570    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stdmh\" (UniqueName: \"kubernetes.io/projected/ea1e6f27-49dc-457b-871f-699f016f9bdf-kube-api-access-stdmh\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.596734 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl7sx\" (UniqueName: \"kubernetes.io/projected/cdcef278-0391-4f9d-b80c-7f404fb55655-kube-api-access-nl7sx\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.596734 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596661    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea1e6f27-49dc-457b-871f-699f016f9bdf-service-ca-bundle\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.596734 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-default-certificate\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.596865 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea1e6f27-49dc-457b-871f-699f016f9bdf-tmp\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.596865 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.596865 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-stats-auth\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.596865 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.597036 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.596888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea1e6f27-49dc-457b-871f-699f016f9bdf-serving-cert\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.597093 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.597039    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:02.097016054 +0000 UTC m=+44.358015789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : configmap references non-existent config key: service-ca.crt
Jul 23 21:52:01.597471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.597267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea1e6f27-49dc-457b-871f-699f016f9bdf-service-ca-bundle\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.597471 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.597362    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 21:52:01.597471 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.597419    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:02.097402861 +0000 UTC m=+44.358402574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : secret "router-metrics-certs-default" not found
Jul 23 21:52:01.597471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.597422    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ea1e6f27-49dc-457b-871f-699f016f9bdf-snapshots\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.597749 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.597625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea1e6f27-49dc-457b-871f-699f016f9bdf-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.597749 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.597667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea1e6f27-49dc-457b-871f-699f016f9bdf-tmp\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.599451 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.599421    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-stats-auth\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.599617 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.599595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea1e6f27-49dc-457b-871f-699f016f9bdf-serving-cert\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.599866 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.599842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-default-certificate\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.607598 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.607540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stdmh\" (UniqueName: \"kubernetes.io/projected/ea1e6f27-49dc-457b-871f-699f016f9bdf-kube-api-access-stdmh\") pod \"insights-operator-555d9d7885-4k9jn\" (UID: \"ea1e6f27-49dc-457b-871f-699f016f9bdf\") " pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.608209 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.608194    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl7sx\" (UniqueName: \"kubernetes.io/projected/cdcef278-0391-4f9d-b80c-7f404fb55655-kube-api-access-nl7sx\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:01.633317 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.633286    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-4k9jn"
Jul 23 21:52:01.726947 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.726921    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s"]
Jul 23 21:52:01.732713 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:01.731350    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c67a01_6139_4677_b7b1_0cf92777ccfb.slice/crio-af1730ba4c13b448afd4618d09d0bbeaa73df936595ae0362460a38c0cffffd7 WatchSource:0}: Error finding container af1730ba4c13b448afd4618d09d0bbeaa73df936595ae0362460a38c0cffffd7: Status 404 returned error can't find the container with id af1730ba4c13b448afd4618d09d0bbeaa73df936595ae0362460a38c0cffffd7
Jul 23 21:52:01.770910 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.770874    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s9bnx"]
Jul 23 21:52:01.773921 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:01.773894    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d931f74_1cd1_40fb_ba18_c613477a3784.slice/crio-19c05b6f47209f7e58626970413d51071c0b23644df6b086b7eae4f909ab1b35 WatchSource:0}: Error finding container 19c05b6f47209f7e58626970413d51071c0b23644df6b086b7eae4f909ab1b35: Status 404 returned error can't find the container with id 19c05b6f47209f7e58626970413d51071c0b23644df6b086b7eae4f909ab1b35
Jul 23 21:52:01.809129 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.809097    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-4k9jn"]
Jul 23 21:52:01.812563 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:01.812528    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1e6f27_49dc_457b_871f_699f016f9bdf.slice/crio-da635e04c3beaf53f7b22758af7b1cf42dcddd49304f75f9d254a33f2b88f3da WatchSource:0}: Error finding container da635e04c3beaf53f7b22758af7b1cf42dcddd49304f75f9d254a33f2b88f3da: Status 404 returned error can't find the container with id da635e04c3beaf53f7b22758af7b1cf42dcddd49304f75f9d254a33f2b88f3da
Jul 23 21:52:01.865045 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.864945    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:01.900973 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:01.900935    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:01.901118 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.901070    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:01.901177 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:01.901151    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls podName:335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:02.901128737 +0000 UTC m=+45.162128444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n9zrt" (UID: "335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58") : secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:02.001926 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.001888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:02.002117 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.002046    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 21:52:02.002117 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.002108    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls podName:79a5c457-e4c5-4fa6-930f-9b6ca5a8b262 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:03.002093104 +0000 UTC m=+45.263092810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fp62t" (UID: "79a5c457-e4c5-4fa6-930f-9b6ca5a8b262") : secret "samples-operator-tls" not found
Jul 23 21:52:02.004958 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.004934    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:02.099139 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.099103    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:02.102966 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.102946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:02.103310 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.103075    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:02.103310 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.103083    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 21:52:02.103310 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.103133    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:03.103120206 +0000 UTC m=+45.364119911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : secret "router-metrics-certs-default" not found
Jul 23 21:52:02.103310 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.103169    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:03.103158198 +0000 UTC m=+45.364157908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : configmap references non-existent config key: service-ca.crt
Jul 23 21:52:02.224978 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.223176    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:02.316621 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.316587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:02.608129 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.608049    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-4k9jn" event={"ID":"ea1e6f27-49dc-457b-871f-699f016f9bdf","Type":"ContainerStarted","Data":"da635e04c3beaf53f7b22758af7b1cf42dcddd49304f75f9d254a33f2b88f3da"}
Jul 23 21:52:02.609712 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.609627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" event={"ID":"9d931f74-1cd1-40fb-ba18-c613477a3784","Type":"ContainerStarted","Data":"19c05b6f47209f7e58626970413d51071c0b23644df6b086b7eae4f909ab1b35"}
Jul 23 21:52:02.611105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.611078    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s" event={"ID":"09c67a01-6139-4677-b7b1-0cf92777ccfb","Type":"ContainerStarted","Data":"af1730ba4c13b448afd4618d09d0bbeaa73df936595ae0362460a38c0cffffd7"}
Jul 23 21:52:02.678764 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.678618    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:02.910563 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:02.910454    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:02.910729 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.910648    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:02.910729 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:02.910728    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls podName:335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:04.91070628 +0000 UTC m=+47.171705989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n9zrt" (UID: "335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58") : secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:03.011204 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.011166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:03.011370 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:03.011330    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 21:52:03.011456 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:03.011419    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls podName:79a5c457-e4c5-4fa6-930f-9b6ca5a8b262 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:05.011394286 +0000 UTC m=+47.272394009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fp62t" (UID: "79a5c457-e4c5-4fa6-930f-9b6ca5a8b262") : secret "samples-operator-tls" not found
Jul 23 21:52:03.057764 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.057726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:03.112437 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.112399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:03.112884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.112461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:03.112884 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:03.112588    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:05.11256433 +0000 UTC m=+47.373564051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : configmap references non-existent config key: service-ca.crt
Jul 23 21:52:03.112884 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:03.112627    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 21:52:03.112884 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:03.112728    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:05.112710276 +0000 UTC m=+47.373709988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : secret "router-metrics-certs-default" not found
Jul 23 21:52:03.118300 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.118266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:03.615317 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.615275    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s" event={"ID":"09c67a01-6139-4677-b7b1-0cf92777ccfb","Type":"ContainerStarted","Data":"92509604b686c0e3f34bca413cca5390b9eced9c35f505a2ef44453ce4912f59"}
Jul 23 21:52:03.632402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.632297    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-67d5s" podStartSLOduration=0.98157847 podStartE2EDuration="2.632279385s" totalImagesPullingTime="1.650700915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:03.631921804 +0000 UTC m=+45.892921523" watchObservedRunningTime="2026-07-23 21:52:03.632279385 +0000 UTC m=+45.893279091"
Jul 23 21:52:03.732416 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.732372    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:03.801957 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.801923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:03.900303 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.900196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:03.975989 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:03.975943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:04.148742 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.148708    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"]
Jul 23 21:52:04.164725 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.164621    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"]
Jul 23 21:52:04.164885 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.164748    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.167104 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.167047    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 23 21:52:04.167104 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.167084    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 23 21:52:04.167291 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.167172    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ncmgd\""
Jul 23 21:52:04.168160 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.167931    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:52:04.168160 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.167976    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 23 21:52:04.264097 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.264057    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:04.322232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.322165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:04.324028 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.324008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvdqx\" (UniqueName: \"kubernetes.io/projected/b22450ac-7815-41e8-954d-d065f8a75de5-kube-api-access-mvdqx\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.324126 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.324044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b22450ac-7815-41e8-954d-d065f8a75de5-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.324205 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.324183    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b22450ac-7815-41e8-954d-d065f8a75de5-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.425012 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.424923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvdqx\" (UniqueName: \"kubernetes.io/projected/b22450ac-7815-41e8-954d-d065f8a75de5-kube-api-access-mvdqx\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.425012 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.424995    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b22450ac-7815-41e8-954d-d065f8a75de5-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.425296 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.425137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b22450ac-7815-41e8-954d-d065f8a75de5-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.425819 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.425797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b22450ac-7815-41e8-954d-d065f8a75de5-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.427671 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.427646    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b22450ac-7815-41e8-954d-d065f8a75de5-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.434953 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.434929    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvdqx\" (UniqueName: \"kubernetes.io/projected/b22450ac-7815-41e8-954d-d065f8a75de5-kube-api-access-mvdqx\") pod \"kube-storage-version-migrator-operator-7658d9b749-95fgg\" (UID: \"b22450ac-7815-41e8-954d-d065f8a75de5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.475961 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.475915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"
Jul 23 21:52:04.928957 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:04.928918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:04.929120 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:04.929029    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:04.929120 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:04.929081    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls podName:335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:08.929067561 +0000 UTC m=+51.190067267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n9zrt" (UID: "335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58") : secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:05.029750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.029715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:05.031057 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:05.030659    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 21:52:05.031057 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:05.030738    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls podName:79a5c457-e4c5-4fa6-930f-9b6ca5a8b262 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:09.030716725 +0000 UTC m=+51.291716431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fp62t" (UID: "79a5c457-e4c5-4fa6-930f-9b6ca5a8b262") : secret "samples-operator-tls" not found
Jul 23 21:52:05.110257 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.110226    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg"]
Jul 23 21:52:05.112910 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:05.112881    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22450ac_7815_41e8_954d_d065f8a75de5.slice/crio-064cfb6f33328a55e92e73955c8857152bae013d9204c3258951cebb9faded6a WatchSource:0}: Error finding container 064cfb6f33328a55e92e73955c8857152bae013d9204c3258951cebb9faded6a: Status 404 returned error can't find the container with id 064cfb6f33328a55e92e73955c8857152bae013d9204c3258951cebb9faded6a
Jul 23 21:52:05.130260 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.130236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:05.130367 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.130275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:05.130423 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:05.130390    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:09.130372897 +0000 UTC m=+51.391372604 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : configmap references non-existent config key: service-ca.crt
Jul 23 21:52:05.130488 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:05.130468    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 21:52:05.130545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:05.130535    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:09.130516568 +0000 UTC m=+51.391516283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : secret "router-metrics-certs-default" not found
Jul 23 21:52:05.621965 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.621938    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/0.log"
Jul 23 21:52:05.622384 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.621979    2575 generic.go:356] "Generic (PLEG): container finished" podID="9d931f74-1cd1-40fb-ba18-c613477a3784" containerID="66398c15d0f3994853b780f2dcd3beac502f35c73f37c558327a1a62d3b6561e" exitCode=255
Jul 23 21:52:05.622384 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.622061    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" event={"ID":"9d931f74-1cd1-40fb-ba18-c613477a3784","Type":"ContainerDied","Data":"66398c15d0f3994853b780f2dcd3beac502f35c73f37c558327a1a62d3b6561e"}
Jul 23 21:52:05.622384 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.622352    2575 scope.go:117] "RemoveContainer" containerID="66398c15d0f3994853b780f2dcd3beac502f35c73f37c558327a1a62d3b6561e"
Jul 23 21:52:05.623669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.623625    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg" event={"ID":"b22450ac-7815-41e8-954d-d065f8a75de5","Type":"ContainerStarted","Data":"064cfb6f33328a55e92e73955c8857152bae013d9204c3258951cebb9faded6a"}
Jul 23 21:52:05.625691 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.625664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-4k9jn" event={"ID":"ea1e6f27-49dc-457b-871f-699f016f9bdf","Type":"ContainerStarted","Data":"803bd05dba5086e529f9343476f36768a1530246418377511dbdba1336ab2487"}
Jul 23 21:52:05.659986 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:05.659929    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-4k9jn" podStartSLOduration=1.481311099 podStartE2EDuration="4.659907198s" totalImagesPullingTime="3.178596099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:05.658340552 +0000 UTC m=+47.919340303" watchObservedRunningTime="2026-07-23 21:52:05.659907198 +0000 UTC m=+47.920906927"
Jul 23 21:52:06.150435 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.150396    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"]
Jul 23 21:52:06.171235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.171201    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"]
Jul 23 21:52:06.171402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.171329    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.173981 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.173918    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 23 21:52:06.173981 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.173935    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:52:06.173981 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.173981    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 23 21:52:06.174215 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.174027    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qlqbt\""
Jul 23 21:52:06.174215 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.173921    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 23 21:52:06.240223 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.240195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11a9d26c-98ae-41b9-bd98-e06274f9d004-config\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.240376 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.240333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9vb\" (UniqueName: \"kubernetes.io/projected/11a9d26c-98ae-41b9-bd98-e06274f9d004-kube-api-access-pl9vb\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.240376 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.240368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11a9d26c-98ae-41b9-bd98-e06274f9d004-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.340770 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.340737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9vb\" (UniqueName: \"kubernetes.io/projected/11a9d26c-98ae-41b9-bd98-e06274f9d004-kube-api-access-pl9vb\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.340923 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.340783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11a9d26c-98ae-41b9-bd98-e06274f9d004-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.340923 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.340819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11a9d26c-98ae-41b9-bd98-e06274f9d004-config\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.341375 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.341351    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11a9d26c-98ae-41b9-bd98-e06274f9d004-config\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.343178 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.343147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11a9d26c-98ae-41b9-bd98-e06274f9d004-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.349450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.349427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9vb\" (UniqueName: \"kubernetes.io/projected/11a9d26c-98ae-41b9-bd98-e06274f9d004-kube-api-access-pl9vb\") pod \"service-ca-operator-5bcbc94c78-hdc7n\" (UID: \"11a9d26c-98ae-41b9-bd98-e06274f9d004\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.480371 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.480291    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"
Jul 23 21:52:06.621424 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.621391    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n"]
Jul 23 21:52:06.624536 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:06.624507    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a9d26c_98ae_41b9_bd98_e06274f9d004.slice/crio-5bf2afcb6b6b9d8fab059fa9f313f36678aa74b0823110e7932fd56e809dd426 WatchSource:0}: Error finding container 5bf2afcb6b6b9d8fab059fa9f313f36678aa74b0823110e7932fd56e809dd426: Status 404 returned error can't find the container with id 5bf2afcb6b6b9d8fab059fa9f313f36678aa74b0823110e7932fd56e809dd426
Jul 23 21:52:06.629323 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.629287    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n" event={"ID":"11a9d26c-98ae-41b9-bd98-e06274f9d004","Type":"ContainerStarted","Data":"5bf2afcb6b6b9d8fab059fa9f313f36678aa74b0823110e7932fd56e809dd426"}
Jul 23 21:52:06.630957 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.630930    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 21:52:06.631322 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.631307    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/0.log"
Jul 23 21:52:06.631415 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.631339    2575 generic.go:356] "Generic (PLEG): container finished" podID="9d931f74-1cd1-40fb-ba18-c613477a3784" containerID="5c1e2fb13839a4fea00019b8ed4004e5e70553409885936c6ddda4ad04aa038c" exitCode=255
Jul 23 21:52:06.631497 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.631424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" event={"ID":"9d931f74-1cd1-40fb-ba18-c613477a3784","Type":"ContainerDied","Data":"5c1e2fb13839a4fea00019b8ed4004e5e70553409885936c6ddda4ad04aa038c"}
Jul 23 21:52:06.631497 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.631478    2575 scope.go:117] "RemoveContainer" containerID="66398c15d0f3994853b780f2dcd3beac502f35c73f37c558327a1a62d3b6561e"
Jul 23 21:52:06.631700 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:06.631683    2575 scope.go:117] "RemoveContainer" containerID="5c1e2fb13839a4fea00019b8ed4004e5e70553409885936c6ddda4ad04aa038c"
Jul 23 21:52:06.631928 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:06.631896    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s9bnx_openshift-console-operator(9d931f74-1cd1-40fb-ba18-c613477a3784)\"" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" podUID="9d931f74-1cd1-40fb-ba18-c613477a3784"
Jul 23 21:52:07.556856 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.556816    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d4tk2"]
Jul 23 21:52:07.578950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.578916    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d4tk2"]
Jul 23 21:52:07.579074 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.579027    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.581356 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.581318    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 21:52:07.581488 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.581319    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 21:52:07.581488 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.581400    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jpf7h\""
Jul 23 21:52:07.586676 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.586619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:07.635392 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.635369    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 21:52:07.635775 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.635718    2575 scope.go:117] "RemoveContainer" containerID="5c1e2fb13839a4fea00019b8ed4004e5e70553409885936c6ddda4ad04aa038c"
Jul 23 21:52:07.635894 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:07.635874    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s9bnx_openshift-console-operator(9d931f74-1cd1-40fb-ba18-c613477a3784)\"" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" podUID="9d931f74-1cd1-40fb-ba18-c613477a3784"
Jul 23 21:52:07.652164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.652130    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d885894-adcb-4801-aabc-8618d6acb004-data-volume\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.652316 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.652239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2whd4\" (UniqueName: \"kubernetes.io/projected/9d885894-adcb-4801-aabc-8618d6acb004-kube-api-access-2whd4\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.652420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.652400    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.652555 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.652527    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d885894-adcb-4801-aabc-8618d6acb004-crio-socket\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.652656 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.652572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.661562 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.661531    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:07.753921 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.753887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.754091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.753965    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d885894-adcb-4801-aabc-8618d6acb004-crio-socket\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.754091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.754001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.754091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.754037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d885894-adcb-4801-aabc-8618d6acb004-data-volume\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.754091 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:07.754050    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:07.754091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.754087    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2whd4\" (UniqueName: \"kubernetes.io/projected/9d885894-adcb-4801-aabc-8618d6acb004-kube-api-access-2whd4\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.754321 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:07.754115    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:08.254094508 +0000 UTC m=+50.515094222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:07.754321 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:07.754136    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 21:52:07.754321 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.754184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d885894-adcb-4801-aabc-8618d6acb004-crio-socket\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.754321 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:07.754230    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:08.254208807 +0000 UTC m=+50.515208523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : configmap "kube-rbac-proxy" not found
Jul 23 21:52:07.754527 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.754458    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d885894-adcb-4801-aabc-8618d6acb004-data-volume\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.767818 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.767789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2whd4\" (UniqueName: \"kubernetes.io/projected/9d885894-adcb-4801-aabc-8618d6acb004-kube-api-access-2whd4\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:07.983825 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.983783    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"]
Jul 23 21:52:07.996164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.996130    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"
Jul 23 21:52:07.996776 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.996715    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"]
Jul 23 21:52:07.998486 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:07.998458    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fnvqb\""
Jul 23 21:52:08.056988 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.056937    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95cvw\" (UniqueName: \"kubernetes.io/projected/93400b4d-aef5-4f0f-96b4-0cfbcdf7b442-kube-api-access-95cvw\") pod \"network-check-source-7bc5b6bf85-nl4nv\" (UID: \"93400b4d-aef5-4f0f-96b4-0cfbcdf7b442\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"
Jul 23 21:52:08.158100 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.158057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95cvw\" (UniqueName: \"kubernetes.io/projected/93400b4d-aef5-4f0f-96b4-0cfbcdf7b442-kube-api-access-95cvw\") pod \"network-check-source-7bc5b6bf85-nl4nv\" (UID: \"93400b4d-aef5-4f0f-96b4-0cfbcdf7b442\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"
Jul 23 21:52:08.169116 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.169082    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95cvw\" (UniqueName: \"kubernetes.io/projected/93400b4d-aef5-4f0f-96b4-0cfbcdf7b442-kube-api-access-95cvw\") pod \"network-check-source-7bc5b6bf85-nl4nv\" (UID: \"93400b4d-aef5-4f0f-96b4-0cfbcdf7b442\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"
Jul 23 21:52:08.258720 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.258615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:08.258896 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.258724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:08.258896 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.258761    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:08.258896 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.258811    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 21:52:08.258896 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.258835    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:09.258821024 +0000 UTC m=+51.519820730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:08.258896 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.258855    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:09.258844047 +0000 UTC m=+51.519843752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : configmap "kube-rbac-proxy" not found
Jul 23 21:52:08.307227 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.307191    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"
Jul 23 21:52:08.458170 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.458145    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv"]
Jul 23 21:52:08.460916 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:08.460877    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93400b4d_aef5_4f0f_96b4_0cfbcdf7b442.slice/crio-cea2b3bda9194d6c60bbf72f852c48f9e548e5c741baeff40f726b71919cc275 WatchSource:0}: Error finding container cea2b3bda9194d6c60bbf72f852c48f9e548e5c741baeff40f726b71919cc275: Status 404 returned error can't find the container with id cea2b3bda9194d6c60bbf72f852c48f9e548e5c741baeff40f726b71919cc275
Jul 23 21:52:08.639648 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.639610    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg" event={"ID":"b22450ac-7815-41e8-954d-d065f8a75de5","Type":"ContainerStarted","Data":"5ac12461f948467b026bba3271f9003206285772369798fabb387549afe4f77d"}
Jul 23 21:52:08.640749 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.640726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv" event={"ID":"93400b4d-aef5-4f0f-96b4-0cfbcdf7b442","Type":"ContainerStarted","Data":"cea2b3bda9194d6c60bbf72f852c48f9e548e5c741baeff40f726b71919cc275"}
Jul 23 21:52:08.658455 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.658369    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg" podStartSLOduration=2.097459856 podStartE2EDuration="4.658348964s" totalImagesPullingTime="2.560889108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:08.657556313 +0000 UTC m=+50.918556069" watchObservedRunningTime="2026-07-23 21:52:08.658348964 +0000 UTC m=+50.919348696"
Jul 23 21:52:08.949145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.949063    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sftqj_d8b2d5df-b34f-4985-b490-555ffb020fcc/dns-node-resolver/0.log"
Jul 23 21:52:08.966597 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.966558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:08.966772 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.966711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:08.966772 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966734    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 21:52:08.966772 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966752    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f864dbdd-2s7xr: secret "image-registry-tls" not found
Jul 23 21:52:08.966772 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:08.966760    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:08.966990 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966811    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls podName:34152caf-86bd-4ff2-8f02-7ea46621d1af nodeName:}" failed. No retries permitted until 2026-07-23 21:52:24.966790659 +0000 UTC m=+67.227790382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls") pod "image-registry-74f864dbdd-2s7xr" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af") : secret "image-registry-tls" not found
Jul 23 21:52:08.966990 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966859    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 21:52:08.966990 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966874    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:08.966990 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966919    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls podName:335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:16.966908234 +0000 UTC m=+59.227907940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n9zrt" (UID: "335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58") : secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:08.966990 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:08.966937    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls podName:27d52fef-85f6-496c-b787-1f334f4c3f33 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:24.966928494 +0000 UTC m=+67.227928203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls") pod "dns-default-v6d4c" (UID: "27d52fef-85f6-496c-b787-1f334f4c3f33") : secret "dns-default-metrics-tls" not found
Jul 23 21:52:09.068586 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.068158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:09.068586 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.068357    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 21:52:09.068586 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.068411    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls podName:79a5c457-e4c5-4fa6-930f-9b6ca5a8b262 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:17.068396519 +0000 UTC m=+59.329396243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fp62t" (UID: "79a5c457-e4c5-4fa6-930f-9b6ca5a8b262") : secret "samples-operator-tls" not found
Jul 23 21:52:09.168980 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.168939    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:09.169148 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.169001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:09.169223 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.169174    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 21:52:09.169275 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.169241    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:17.16922267 +0000 UTC m=+59.430222377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : secret "router-metrics-certs-default" not found
Jul 23 21:52:09.169355 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.169322    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:17.169304246 +0000 UTC m=+59.430303974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : configmap references non-existent config key: service-ca.crt
Jul 23 21:52:09.225442 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.225335    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:09.270664 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.270608    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:09.270827 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.270738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:09.271070 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.271003    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:09.271205 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:09.271082    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.271062563 +0000 UTC m=+53.532062272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:09.271408 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.271384    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d885894-adcb-4801-aabc-8618d6acb004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:09.948590 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:09.948561    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz8gs_b2d08fc9-7fee-4c81-9b23-ffad61e60a7b/node-ca/0.log"
Jul 23 21:52:10.360548 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.360523    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hjj5g_ddf53267-738e-4f2f-852b-9ea39603709c/serve-healthcheck-canary/0.log"
Jul 23 21:52:10.363102 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.363067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:10.373402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.373377    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"]
Jul 23 21:52:10.392398 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.392191    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"]
Jul 23 21:52:10.392398 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.392313    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"
Jul 23 21:52:10.395474 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.395450    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xxfkw\""
Jul 23 21:52:10.395746 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.395725    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 23 21:52:10.396235 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.396197    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 23 21:52:10.467185 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.467149    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:10.483327 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.483294    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmjd\" (UniqueName: \"kubernetes.io/projected/e46ed525-9024-4013-9810-e6c7972fb437-kube-api-access-htmjd\") pod \"migrator-7cc9c7ffd7-vs5vk\" (UID: \"e46ed525-9024-4013-9810-e6c7972fb437\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"
Jul 23 21:52:10.563848 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.563796    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:10.584010 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.583977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmjd\" (UniqueName: \"kubernetes.io/projected/e46ed525-9024-4013-9810-e6c7972fb437-kube-api-access-htmjd\") pod \"migrator-7cc9c7ffd7-vs5vk\" (UID: \"e46ed525-9024-4013-9810-e6c7972fb437\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"
Jul 23 21:52:10.592598 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.592565    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmjd\" (UniqueName: \"kubernetes.io/projected/e46ed525-9024-4013-9810-e6c7972fb437-kube-api-access-htmjd\") pod \"migrator-7cc9c7ffd7-vs5vk\" (UID: \"e46ed525-9024-4013-9810-e6c7972fb437\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"
Jul 23 21:52:10.649372 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.649290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n" event={"ID":"11a9d26c-98ae-41b9-bd98-e06274f9d004","Type":"ContainerStarted","Data":"a03f1d9baa36e113e515b1241988a9ae34390e5c5f43556ceec2e85f2811df11"}
Jul 23 21:52:10.668054 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.667985    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n" podStartSLOduration=1.27272738 podStartE2EDuration="4.667965593s" totalImagesPullingTime="3.395238213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:10.666166692 +0000 UTC m=+52.927166422" watchObservedRunningTime="2026-07-23 21:52:10.667965593 +0000 UTC m=+52.928965321"
Jul 23 21:52:10.705081 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:10.705042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"
Jul 23 21:52:11.291489 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.291454    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:11.291960 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:11.291627    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:11.291960 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:11.291727    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:15.291703291 +0000 UTC m=+57.552702996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:11.353023 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.352986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:11.432750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.432695    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:11.529571 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.529533    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:11.585231 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.585022    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:11.585231 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.585075    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:11.585534 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.585516    2575 scope.go:117] "RemoveContainer" containerID="5c1e2fb13839a4fea00019b8ed4004e5e70553409885936c6ddda4ad04aa038c"
Jul 23 21:52:11.585782 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:11.585759    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s9bnx_openshift-console-operator(9d931f74-1cd1-40fb-ba18-c613477a3784)\"" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" podUID="9d931f74-1cd1-40fb-ba18-c613477a3784"
Jul 23 21:52:11.595093 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.595065    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:11.645202 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.645182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk"]
Jul 23 21:52:11.647744 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:11.647714    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46ed525_9024_4013_9810_e6c7972fb437.slice/crio-d76678f7c9c45e98a64402924bd2ba15a3aca40bf8fd6555ab6c89360064bad3 WatchSource:0}: Error finding container d76678f7c9c45e98a64402924bd2ba15a3aca40bf8fd6555ab6c89360064bad3: Status 404 returned error can't find the container with id d76678f7c9c45e98a64402924bd2ba15a3aca40bf8fd6555ab6c89360064bad3
Jul 23 21:52:11.653391 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.653365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk" event={"ID":"e46ed525-9024-4013-9810-e6c7972fb437","Type":"ContainerStarted","Data":"d76678f7c9c45e98a64402924bd2ba15a3aca40bf8fd6555ab6c89360064bad3"}
Jul 23 21:52:11.775588 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.775552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:11.887792 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:11.887767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:12.051184 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:12.051148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:12.149736 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:12.149661    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:12.658377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:12.658338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv" event={"ID":"93400b4d-aef5-4f0f-96b4-0cfbcdf7b442","Type":"ContainerStarted","Data":"a56e916bd93066efc4436aeb15f428e3996f305eb6dfc1866d3ec2a1b1de5e7b"}
Jul 23 21:52:12.687326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:12.687263    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nl4nv" podStartSLOduration=2.260968403 podStartE2EDuration="5.687247737s" totalImagesPullingTime="3.426279334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:12.686084284 +0000 UTC m=+54.947084013" watchObservedRunningTime="2026-07-23 21:52:12.687247737 +0000 UTC m=+54.948247529"
Jul 23 21:52:13.663028 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:13.663002    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk" event={"ID":"e46ed525-9024-4013-9810-e6c7972fb437","Type":"ContainerStarted","Data":"f9a7f1dce80fa23995504b8a06468de9301c921bb4141ea00e79e04911ce0add"}
Jul 23 21:52:13.687810 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:13.687774    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:13.753371 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:13.753284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:14.650364 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:14.650329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:14.667755 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:14.667726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk" event={"ID":"e46ed525-9024-4013-9810-e6c7972fb437","Type":"ContainerStarted","Data":"ef379621863c1d09b617adefb0ca0aa2ed171ed3cb12ac3ad6bb28823299c867"}
Jul 23 21:52:14.687849 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:14.687797    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vs5vk" podStartSLOduration=2.84942975 podStartE2EDuration="4.687781354s" totalImagesPullingTime="1.838351604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:14.686734946 +0000 UTC m=+56.947734674" watchObservedRunningTime="2026-07-23 21:52:14.687781354 +0000 UTC m=+56.948781083"
Jul 23 21:52:15.327214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:15.327172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:15.327474 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:15.327288    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:15.327474 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:15.327342    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:23.327328353 +0000 UTC m=+65.588328059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:15.933380 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:15.933346    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:16.051194 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:16.051158    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:16.205438 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:16.205355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:16.306067 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:16.306026    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:17.043609 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:17.043574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:17.044008 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.043738    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:17.044008 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.043808    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls podName:335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.043790781 +0000 UTC m=+75.304790490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n9zrt" (UID: "335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58") : secret "cluster-monitoring-operator-tls" not found
Jul 23 21:52:17.144491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:17.144455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:17.144672 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.144605    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 21:52:17.144718 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.144697    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls podName:79a5c457-e4c5-4fa6-930f-9b6ca5a8b262 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.144681599 +0000 UTC m=+75.405681305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-fp62t" (UID: "79a5c457-e4c5-4fa6-930f-9b6ca5a8b262") : secret "samples-operator-tls" not found
Jul 23 21:52:17.245926 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:17.245887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:17.245926 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:17.245929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:17.246131 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.246027    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 21:52:17.246131 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.246066    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.246047677 +0000 UTC m=+75.507047384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : configmap references non-existent config key: service-ca.crt
Jul 23 21:52:17.246131 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:17.246088    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs podName:cdcef278-0391-4f9d-b80c-7f404fb55655 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.246081161 +0000 UTC m=+75.507080866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs") pod "router-default-7f9b7f69fb-bxs2f" (UID: "cdcef278-0391-4f9d-b80c-7f404fb55655") : secret "router-metrics-certs-default" not found
Jul 23 21:52:18.126094 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:18.126052    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:18.565022 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:18.564944    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wql74"
Jul 23 21:52:18.765118 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:18.765084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:18.998533 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:18.998498    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:19.093954 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:19.093918    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:19.195537 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:19.195497    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:19.295935 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:19.295848    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:19.379378 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:19.379348    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:19.468544 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:19.468507    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:20.451771 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:20.451733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:20.702684 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:20.702553    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:21.049341 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:21.049251    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:21.257257 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:21.257225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:21.660205 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:21.660165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.063903 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.063812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.221316 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.221283    2575 scope.go:117] "RemoveContainer" containerID="5c1e2fb13839a4fea00019b8ed4004e5e70553409885936c6ddda4ad04aa038c"
Jul 23 21:52:22.454057 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.454019    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.623974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.623939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.695654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.695611    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 21:52:22.696027 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.695705    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" event={"ID":"9d931f74-1cd1-40fb-ba18-c613477a3784","Type":"ContainerStarted","Data":"4f3afb88036039c8f293944635276cdeb4fc55c565d42e59ea0fd7afb81ab4b8"}
Jul 23 21:52:22.696027 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.695969    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:22.700339 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.700315    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx"
Jul 23 21:52:22.713667 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.713568    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-s9bnx" podStartSLOduration=18.489840795 podStartE2EDuration="21.713554283s" totalImagesPullingTime="3.223713488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:22.712406171 +0000 UTC m=+64.973405920" watchObservedRunningTime="2026-07-23 21:52:22.713554283 +0000 UTC m=+64.974554011"
Jul 23 21:52:22.716833 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.716809    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.783507 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.783471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.884846 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.884811    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:22.993179 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:22.993075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:23.090836 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.090797    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:23.211440 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.211408    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:23.402191 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.402162    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:23.402361 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:23.402307    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:23.402414 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:23.402367    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls podName:9d885894-adcb-4801-aabc-8618d6acb004 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:39.402351107 +0000 UTC m=+81.663350813 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4tk2" (UID: "9d885894-adcb-4801-aabc-8618d6acb004") : secret "insights-runtime-extractor-tls" not found
Jul 23 21:52:23.448067 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.448036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:23.550567 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.550516    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:23.648222 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.648174    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:23.748705 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:23.748573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.054168 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.054084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.109426 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.109391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:52:24.109596 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.109505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:52:24.112310 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.112294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 21:52:24.112376 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.112332    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 21:52:24.120304 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:24.120282    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 23 21:52:24.120398 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:52:24.120382    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs podName:f6571af9-eab4-4e45-8560-9c0065dcabfd nodeName:}" failed. No retries permitted until 2026-07-23 21:53:28.120361461 +0000 UTC m=+130.381361172 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs") pod "network-metrics-daemon-9t6qd" (UID: "f6571af9-eab4-4e45-8560-9c0065dcabfd") : secret "metrics-daemon-secret" not found
Jul 23 21:52:24.122447 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.122423    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/886ede06-fccf-4379-831a-41c244ab7387-original-pull-secret\") pod \"global-pull-secret-syncer-29qg9\" (UID: \"886ede06-fccf-4379-831a-41c244ab7387\") " pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:52:24.134387 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.134368    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-29qg9"
Jul 23 21:52:24.198137 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.198102    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.210732 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.210701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:52:24.214022 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.213961    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sbds\" (UniqueName: \"kubernetes.io/projected/2a3d3f22-dfbf-4d92-8704-0b27641d38a8-kube-api-access-7sbds\") pod \"network-check-target-tq988\" (UID: \"2a3d3f22-dfbf-4d92-8704-0b27641d38a8\") " pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:52:24.258351 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.258319    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-29qg9"]
Jul 23 21:52:24.261483 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:24.261455    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886ede06_fccf_4379_831a_41c244ab7387.slice/crio-aa681f9957f34ad3ecefa901ad92cf2a5f653e99c45a57558d8a39b4dd9681d0 WatchSource:0}: Error finding container aa681f9957f34ad3ecefa901ad92cf2a5f653e99c45a57558d8a39b4dd9681d0: Status 404 returned error can't find the container with id aa681f9957f34ad3ecefa901ad92cf2a5f653e99c45a57558d8a39b4dd9681d0
Jul 23 21:52:24.261977 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.261948    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.398708 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.398674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.442746 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.442715    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sc2x2\""
Jul 23 21:52:24.450778 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.450750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:52:24.509279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.506588    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.600722 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.600692    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tq988"]
Jul 23 21:52:24.603792 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:24.603767    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3d3f22_dfbf_4d92_8704_0b27641d38a8.slice/crio-1859d9ed63d171a1248d292264878419fd381b475a8b9072f807daf34a2b81c0 WatchSource:0}: Error finding container 1859d9ed63d171a1248d292264878419fd381b475a8b9072f807daf34a2b81c0: Status 404 returned error can't find the container with id 1859d9ed63d171a1248d292264878419fd381b475a8b9072f807daf34a2b81c0
Jul 23 21:52:24.651115 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.651086    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:24.703027 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.702994    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-29qg9" event={"ID":"886ede06-fccf-4379-831a-41c244ab7387","Type":"ContainerStarted","Data":"aa681f9957f34ad3ecefa901ad92cf2a5f653e99c45a57558d8a39b4dd9681d0"}
Jul 23 21:52:24.704370 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.704344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tq988" event={"ID":"2a3d3f22-dfbf-4d92-8704-0b27641d38a8","Type":"ContainerStarted","Data":"02dd9bf5d06e1d3e1db19299568410da8a10596f5c44ea82e1237b5685eebbf6"}
Jul 23 21:52:24.704472 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.704390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tq988" event={"ID":"2a3d3f22-dfbf-4d92-8704-0b27641d38a8","Type":"ContainerStarted","Data":"1859d9ed63d171a1248d292264878419fd381b475a8b9072f807daf34a2b81c0"}
Jul 23 21:52:24.720849 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:24.720798    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tq988" podStartSLOduration=65.720784145 podStartE2EDuration="1m5.720784145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:24.720261404 +0000 UTC m=+66.981261132" watchObservedRunningTime="2026-07-23 21:52:24.720784145 +0000 UTC m=+66.981783900"
Jul 23 21:52:25.019266 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.019184    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:25.019625 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.019272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:25.021480 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.021459    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27d52fef-85f6-496c-b787-1f334f4c3f33-metrics-tls\") pod \"dns-default-v6d4c\" (UID: \"27d52fef-85f6-496c-b787-1f334f4c3f33\") " pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:25.021604 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.021585    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"image-registry-74f864dbdd-2s7xr\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") " pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:25.060291 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.060255    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:25.123311 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.123276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:25.226996 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.226966    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rdwr9\""
Jul 23 21:52:25.235384 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.235322    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:25.237936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.237914    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b7bnz\""
Jul 23 21:52:25.246336 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.246310    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:25.408967 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.408937    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74f864dbdd-2s7xr"]
Jul 23 21:52:25.411537 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.411491    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v6d4c"]
Jul 23 21:52:25.412949 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:25.412925    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34152caf_86bd_4ff2_8f02_7ea46621d1af.slice/crio-67e94496b7a25325d7eb3b41fcd2e2de345c424e22df215d6c2f3ae26b4e9caf WatchSource:0}: Error finding container 67e94496b7a25325d7eb3b41fcd2e2de345c424e22df215d6c2f3ae26b4e9caf: Status 404 returned error can't find the container with id 67e94496b7a25325d7eb3b41fcd2e2de345c424e22df215d6c2f3ae26b4e9caf
Jul 23 21:52:25.414615 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:25.414593    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d52fef_85f6_496c_b787_1f334f4c3f33.slice/crio-37db45623a5334c60ffd4d73071d943afb7474fe7dbc37121914b123d9ac5207 WatchSource:0}: Error finding container 37db45623a5334c60ffd4d73071d943afb7474fe7dbc37121914b123d9ac5207: Status 404 returned error can't find the container with id 37db45623a5334c60ffd4d73071d943afb7474fe7dbc37121914b123d9ac5207
Jul 23 21:52:25.491348 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.491314    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:25.673008 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.672973    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:25.710882 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.710845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6d4c" event={"ID":"27d52fef-85f6-496c-b787-1f334f4c3f33","Type":"ContainerStarted","Data":"37db45623a5334c60ffd4d73071d943afb7474fe7dbc37121914b123d9ac5207"}
Jul 23 21:52:25.712948 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.712912    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" event={"ID":"34152caf-86bd-4ff2-8f02-7ea46621d1af","Type":"ContainerStarted","Data":"5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e"}
Jul 23 21:52:25.713086 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.712966    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" event={"ID":"34152caf-86bd-4ff2-8f02-7ea46621d1af","Type":"ContainerStarted","Data":"67e94496b7a25325d7eb3b41fcd2e2de345c424e22df215d6c2f3ae26b4e9caf"}
Jul 23 21:52:25.713340 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.713317    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:25.713481 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.713356    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:52:25.736261 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.736206    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" podStartSLOduration=66.736190732 podStartE2EDuration="1m6.736190732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:25.735027224 +0000 UTC m=+67.996026952" watchObservedRunningTime="2026-07-23 21:52:25.736190732 +0000 UTC m=+67.997190479"
Jul 23 21:52:25.883348 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:25.883292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:26.249093 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:26.249002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:26.847777 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:26.847727    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:27.060444 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:27.060412    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:27.459097 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:27.459040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:27.653728 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:27.653678    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:27.850650 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:27.850585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:28.847175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:28.847142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:29.050120 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.050081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:29.442968 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.442933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:29.658131 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.658042    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:29.729079 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.729046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-29qg9" event={"ID":"886ede06-fccf-4379-831a-41c244ab7387","Type":"ContainerStarted","Data":"1a16d5b9d391f8c8934be849285d96b6cbe46fc9798734ed6871cdc52786b465"}
Jul 23 21:52:29.730824 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.730794    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6d4c" event={"ID":"27d52fef-85f6-496c-b787-1f334f4c3f33","Type":"ContainerStarted","Data":"e3eb442a8555da8f1902a6bffefbe037ad228a073c47d0aea134fcfc757e3c1a"}
Jul 23 21:52:29.730931 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.730837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v6d4c" event={"ID":"27d52fef-85f6-496c-b787-1f334f4c3f33","Type":"ContainerStarted","Data":"88c30edc33583c02a18fce9c21eb1936a831c77901a2f9d74b08588c619c68b3"}
Jul 23 21:52:29.730931 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.730860    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:29.744925 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.744866    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-29qg9" podStartSLOduration=65.925220986 podStartE2EDuration="1m10.744850703s" totalImagesPullingTime="4.819629717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:29.744664184 +0000 UTC m=+72.005663913" watchObservedRunningTime="2026-07-23 21:52:29.744850703 +0000 UTC m=+72.005850431"
Jul 23 21:52:29.763712 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.763654    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v6d4c" podStartSLOduration=33.109565779 podStartE2EDuration="36.763621368s" totalImagesPullingTime="3.654055589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:29.762676001 +0000 UTC m=+72.023675728" watchObservedRunningTime="2026-07-23 21:52:29.763621368 +0000 UTC m=+72.024621097"
Jul 23 21:52:29.853693 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:29.853656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:31.243650 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:31.243597    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:31.344273 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:31.344237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:33.090031 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.089990    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-rcg2x"]
Jul 23 21:52:33.092989 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.092968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:33.093119 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.093044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:33.095469 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.095446    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n9zrt\" (UID: \"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:33.098819 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.098798    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 21:52:33.101145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.101124    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cbpfl\""
Jul 23 21:52:33.104228 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.104206    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 21:52:33.109612 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.109593    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rcg2x"]
Jul 23 21:52:33.137866 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.137829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:33.193704 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.193663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:33.193888 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.193758    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgl2p\" (UniqueName: \"kubernetes.io/projected/a7c2c9d0-7520-49e3-a45f-2f9a84d08e99-kube-api-access-xgl2p\") pod \"downloads-5bc67dcd46-rcg2x\" (UID: \"a7c2c9d0-7520-49e3-a45f-2f9a84d08e99\") " pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:33.196165 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.196136    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79a5c457-e4c5-4fa6-930f-9b6ca5a8b262-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-fp62t\" (UID: \"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:33.244883 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.244852    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dfcgf\""
Jul 23 21:52:33.251838 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.251807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"
Jul 23 21:52:33.285011 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.284976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:33.294387 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.294359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:33.294541 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.294411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgl2p\" (UniqueName: \"kubernetes.io/projected/a7c2c9d0-7520-49e3-a45f-2f9a84d08e99-kube-api-access-xgl2p\") pod \"downloads-5bc67dcd46-rcg2x\" (UID: \"a7c2c9d0-7520-49e3-a45f-2f9a84d08e99\") " pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:33.294541 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.294471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:33.295101 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.295078    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdcef278-0391-4f9d-b80c-7f404fb55655-service-ca-bundle\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:33.296892 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.296867    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdcef278-0391-4f9d-b80c-7f404fb55655-metrics-certs\") pod \"router-default-7f9b7f69fb-bxs2f\" (UID: \"cdcef278-0391-4f9d-b80c-7f404fb55655\") " pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:33.305625 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.305598    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgl2p\" (UniqueName: \"kubernetes.io/projected/a7c2c9d0-7520-49e3-a45f-2f9a84d08e99-kube-api-access-xgl2p\") pod \"downloads-5bc67dcd46-rcg2x\" (UID: \"a7c2c9d0-7520-49e3-a45f-2f9a84d08e99\") " pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:33.335989 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.335964    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pfn8f\""
Jul 23 21:52:33.344309 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.344282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"
Jul 23 21:52:33.377915 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.377880    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:33.382308 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.382281    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt"]
Jul 23 21:52:33.385911 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:33.385878    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod335b3ca4_7b4a_4e04_b53f_f24a0c7b7e58.slice/crio-cb45977ecbc54e65d29087258dbf3470c34893ab7c8a39c407e5dd01979cbec9 WatchSource:0}: Error finding container cb45977ecbc54e65d29087258dbf3470c34893ab7c8a39c407e5dd01979cbec9: Status 404 returned error can't find the container with id cb45977ecbc54e65d29087258dbf3470c34893ab7c8a39c407e5dd01979cbec9
Jul 23 21:52:33.406887 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.406865    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:33.461242 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.461181    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mqtwk\""
Jul 23 21:52:33.469528 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.469504    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:33.477452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.477423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:33.482104 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.482072    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t"]
Jul 23 21:52:33.549722 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.549699    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rcg2x"]
Jul 23 21:52:33.551611 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:33.551587    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c2c9d0_7520_49e3_a45f_2f9a84d08e99.slice/crio-7ecf4c567a71cf7454a3de47dc9100433ca6a1d3c3afc04cf8037d515299be04 WatchSource:0}: Error finding container 7ecf4c567a71cf7454a3de47dc9100433ca6a1d3c3afc04cf8037d515299be04: Status 404 returned error can't find the container with id 7ecf4c567a71cf7454a3de47dc9100433ca6a1d3c3afc04cf8037d515299be04
Jul 23 21:52:33.559274 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.559248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:33.604188 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.604016    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f9b7f69fb-bxs2f"]
Jul 23 21:52:33.606561 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:33.606536    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdcef278_0391_4f9d_b80c_7f404fb55655.slice/crio-d7d7bd58aad214e884b33347c21a8102abf8fd0924f61ade3e083d6b8b257847 WatchSource:0}: Error finding container d7d7bd58aad214e884b33347c21a8102abf8fd0924f61ade3e083d6b8b257847: Status 404 returned error can't find the container with id d7d7bd58aad214e884b33347c21a8102abf8fd0924f61ade3e083d6b8b257847
Jul 23 21:52:33.745571 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.745527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rcg2x" event={"ID":"a7c2c9d0-7520-49e3-a45f-2f9a84d08e99","Type":"ContainerStarted","Data":"7ecf4c567a71cf7454a3de47dc9100433ca6a1d3c3afc04cf8037d515299be04"}
Jul 23 21:52:33.746817 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.746790    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t" event={"ID":"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262","Type":"ContainerStarted","Data":"5c1c055262fbd9c9d6c602d293c48b0551e37652b1668e7e800a962c34402a1e"}
Jul 23 21:52:33.747887 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.747863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt" event={"ID":"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58","Type":"ContainerStarted","Data":"cb45977ecbc54e65d29087258dbf3470c34893ab7c8a39c407e5dd01979cbec9"}
Jul 23 21:52:33.749179 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.749157    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f" event={"ID":"cdcef278-0391-4f9d-b80c-7f404fb55655","Type":"ContainerStarted","Data":"060f09f0c6a7ad6b04fcb598945979a6b1c23a4386a2d17904c5c9fdf818ceda"}
Jul 23 21:52:33.749264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:33.749199    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f" event={"ID":"cdcef278-0391-4f9d-b80c-7f404fb55655","Type":"ContainerStarted","Data":"d7d7bd58aad214e884b33347c21a8102abf8fd0924f61ade3e083d6b8b257847"}
Jul 23 21:52:34.470363 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:34.470327    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:34.473684 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:34.473537    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:34.498705 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:34.496591    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f" podStartSLOduration=33.496570355 podStartE2EDuration="33.496570355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:33.771555924 +0000 UTC m=+76.032555662" watchObservedRunningTime="2026-07-23 21:52:34.496570355 +0000 UTC m=+76.757570084"
Jul 23 21:52:34.755096 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:34.754973    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:34.756501 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:34.756299    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f9b7f69fb-bxs2f"
Jul 23 21:52:36.508746 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.508528    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:36.583962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.583913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:36.719420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.719376    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:36.766354 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.766259    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t" event={"ID":"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262","Type":"ContainerStarted","Data":"674fd0cba1a36729fb220928e26463a309019dfd5b6275ffe82f01832f1ea36d"}
Jul 23 21:52:36.766354 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.766325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t" event={"ID":"79a5c457-e4c5-4fa6-930f-9b6ca5a8b262","Type":"ContainerStarted","Data":"1211800e0ba51b3d96bed2ac4acb8463529b0bf858799a40d756ea0c6faa20f9"}
Jul 23 21:52:36.768700 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.768671    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt" event={"ID":"335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58","Type":"ContainerStarted","Data":"5765b56624d2d8bca0b5752768b25676618dde6aef8c9027326857efb19af50c"}
Jul 23 21:52:36.788311 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.788256    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-fp62t" podStartSLOduration=33.133387291 podStartE2EDuration="35.788237957s" totalImagesPullingTime="2.654850666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:36.786552116 +0000 UTC m=+79.047551863" watchObservedRunningTime="2026-07-23 21:52:36.788237957 +0000 UTC m=+79.049237686"
Jul 23 21:52:36.804170 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.804120    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n9zrt" podStartSLOduration=32.988056361 podStartE2EDuration="35.804107901s" totalImagesPullingTime="2.81605154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:36.803649021 +0000 UTC m=+79.064648744" watchObservedRunningTime="2026-07-23 21:52:36.804107901 +0000 UTC m=+79.065107628"
Jul 23 21:52:36.822389 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.822358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:36.902832 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.902797    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:36.907082 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.907048    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"]
Jul 23 21:52:36.927661 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.927611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"]
Jul 23 21:52:36.927813 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.927750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:36.930383 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.930361    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 23 21:52:36.930812 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:36.930791    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lkqfd\""
Jul 23 21:52:37.027366 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.027277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/62a1ff30-1fbe-4e62-8287-a42b865c42a6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-s7nc6\" (UID: \"62a1ff30-1fbe-4e62-8287-a42b865c42a6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:37.040228 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.040184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:37.127853 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.127816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/62a1ff30-1fbe-4e62-8287-a42b865c42a6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-s7nc6\" (UID: \"62a1ff30-1fbe-4e62-8287-a42b865c42a6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:37.130807 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.130760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/62a1ff30-1fbe-4e62-8287-a42b865c42a6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-s7nc6\" (UID: \"62a1ff30-1fbe-4e62-8287-a42b865c42a6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:37.238954 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.238919    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:37.413750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.413720    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"]
Jul 23 21:52:37.416703 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:37.416667    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a1ff30_1fbe_4e62_8287_a42b865c42a6.slice/crio-98366aeac455543684f72ecabfd87bbff5c9444081c4339d87af64dc29026c26 WatchSource:0}: Error finding container 98366aeac455543684f72ecabfd87bbff5c9444081c4339d87af64dc29026c26: Status 404 returned error can't find the container with id 98366aeac455543684f72ecabfd87bbff5c9444081c4339d87af64dc29026c26
Jul 23 21:52:37.773599 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.773558    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6" event={"ID":"62a1ff30-1fbe-4e62-8287-a42b865c42a6","Type":"ContainerStarted","Data":"98366aeac455543684f72ecabfd87bbff5c9444081c4339d87af64dc29026c26"}
Jul 23 21:52:37.969212 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:37.969164    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:38.045975 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.045895    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:38.507391 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.507295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:38.571129 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.571080    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:38.779596 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.779492    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6" event={"ID":"62a1ff30-1fbe-4e62-8287-a42b865c42a6","Type":"ContainerStarted","Data":"1bd306af6b0527721fd195a67cd0a7124219ac28538ab023dcbd28610ae686f6"}
Jul 23 21:52:38.780055 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.779858    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:38.781680 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.781617    2575 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-85574b5f49-s7nc6 container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.21:8443/healthz\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body=
Jul 23 21:52:38.781821 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.781708    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6" podUID="62a1ff30-1fbe-4e62-8287-a42b865c42a6" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.21:8443/healthz\": dial tcp 10.132.0.21:8443: connect: connection refused"
Jul 23 21:52:38.794773 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:38.794730    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6" podStartSLOduration=1.527813654 podStartE2EDuration="2.794717755s" totalImagesPullingTime="1.266904101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:38.794198926 +0000 UTC m=+81.055198655" watchObservedRunningTime="2026-07-23 21:52:38.794717755 +0000 UTC m=+81.055717482"
Jul 23 21:52:39.132312 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.132241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:39.202129 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.202088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:39.449954 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.449866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:39.452592 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.452559    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d885894-adcb-4801-aabc-8618d6acb004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4tk2\" (UID: \"9d885894-adcb-4801-aabc-8618d6acb004\") " pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:39.691141 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.691107    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jpf7h\""
Jul 23 21:52:39.699946 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.699915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d4tk2"
Jul 23 21:52:39.736255 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.735718    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v6d4c"
Jul 23 21:52:39.790929 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.790890    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-s7nc6"
Jul 23 21:52:39.859399 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.859226    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d4tk2"]
Jul 23 21:52:39.862664 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:39.862618    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d885894_adcb_4801_aabc_8618d6acb004.slice/crio-9f39260cf2fc34f3d4d4bea6fd8fb03ef1783800985511a25a9767c9a9176879 WatchSource:0}: Error finding container 9f39260cf2fc34f3d4d4bea6fd8fb03ef1783800985511a25a9767c9a9176879: Status 404 returned error can't find the container with id 9f39260cf2fc34f3d4d4bea6fd8fb03ef1783800985511a25a9767c9a9176879
Jul 23 21:52:39.938227 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:39.938181    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:40.017293 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:40.017200    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:40.227849 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:40.227813    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:40.691245 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:40.691100    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:40.791680 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:40.791627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4tk2" event={"ID":"9d885894-adcb-4801-aabc-8618d6acb004","Type":"ContainerStarted","Data":"9f39260cf2fc34f3d4d4bea6fd8fb03ef1783800985511a25a9767c9a9176879"}
Jul 23 21:52:41.044196 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:41.044117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:41.145450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:41.145409    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:42.114598 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:42.114540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:42.219091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:42.219046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:42.376289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:42.376208    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:42.445079 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:42.444971    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:43.063050 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:43.062960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:43.162610 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:43.162567    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:43.820146 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:43.820106    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:43.921415 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:43.921365    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:44.152909 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:44.152842    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:44.250675 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:44.250530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:45.240648 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:45.240590    2575 patch_prober.go:28] interesting pod/image-registry-74f864dbdd-2s7xr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 21:52:45.240823 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:45.240666    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" podUID="34152caf-86bd-4ff2-8f02-7ea46621d1af" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 21:52:45.979329 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:45.979287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:46.201683 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:46.201616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:46.261146 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:46.261031    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:46.723271 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:46.723239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:52:46.993422 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:46.993329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:47.094450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:47.094411    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:48.422937 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:48.422899    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:48.910535 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:48.910500    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:48.974030 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:48.973985    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:49.191186 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:49.191035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:49.247452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:49.247415    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:50.188477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:50.188444    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:50.243402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:50.243366    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.139815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.139772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.314490 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.314451    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.423330 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.423240    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.477435 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.477400    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cb9xj"]
Jul 23 21:52:51.484381 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.484349    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.487182 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.487151    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 21:52:51.487935 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.487906    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 21:52:51.488563 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.488215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 21:52:51.488563 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.488220    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gzhmv\""
Jul 23 21:52:51.488563 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.488360    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 21:52:51.515993 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.515940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.555825 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.555792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-wtmp\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.555993 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.555830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-root\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.555993 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.555860    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-tls\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.555993 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.555977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.556121 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.556008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-accelerators-collector-config\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.556121 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.556033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-textfile\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.556121 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.556059    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-sys\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.556121 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.556096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f56cb8fc-c371-4fc0-998a-843ff6237672-metrics-client-ca\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.556279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.556128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wc4l\" (UniqueName: \"kubernetes.io/projected/f56cb8fc-c371-4fc0-998a-843ff6237672-kube-api-access-8wc4l\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.615729 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.615678    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.656552 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656552 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-accelerators-collector-config\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-textfile\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-sys\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f56cb8fc-c371-4fc0-998a-843ff6237672-metrics-client-ca\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wc4l\" (UniqueName: \"kubernetes.io/projected/f56cb8fc-c371-4fc0-998a-843ff6237672-kube-api-access-8wc4l\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-wtmp\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656755    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-root\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656768    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-sys\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.656816 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.656780    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-tls\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.657202 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.657010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-wtmp\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.657202 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.657087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-textfile\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.657202 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.657096    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f56cb8fc-c371-4fc0-998a-843ff6237672-root\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.657345 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.657222    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-accelerators-collector-config\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.657345 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.657250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f56cb8fc-c371-4fc0-998a-843ff6237672-metrics-client-ca\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.659246 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.659223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.659736 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.659713    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f56cb8fc-c371-4fc0-998a-843ff6237672-node-exporter-tls\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.665914 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.665876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wc4l\" (UniqueName: \"kubernetes.io/projected/f56cb8fc-c371-4fc0-998a-843ff6237672-kube-api-access-8wc4l\") pod \"node-exporter-cb9xj\" (UID: \"f56cb8fc-c371-4fc0-998a-843ff6237672\") " pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.712019 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.711895    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.796711 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.796674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cb9xj"
Jul 23 21:52:51.812471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.812395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:51.912896 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:51.912834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:52.012917 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:52.012815    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:52.313718 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:52.313607    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:52.423649 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:52.423595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:52.867011 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:52.866972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:53.027996 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:52:53.027952    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56cb8fc_c371_4fc0_998a_843ff6237672.slice/crio-8a32eb2d9be7c444480540688efe5143aae8800b1ee7aa5c24973a2c72841ec1 WatchSource:0}: Error finding container 8a32eb2d9be7c444480540688efe5143aae8800b1ee7aa5c24973a2c72841ec1: Status 404 returned error can't find the container with id 8a32eb2d9be7c444480540688efe5143aae8800b1ee7aa5c24973a2c72841ec1
Jul 23 21:52:53.849674 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:53.849022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rcg2x" event={"ID":"a7c2c9d0-7520-49e3-a45f-2f9a84d08e99","Type":"ContainerStarted","Data":"ef9929e70b902b53b27e151a86f957bde69bcedab5b9a68ca5ccad543fa14945"}
Jul 23 21:52:53.852330 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:53.850924    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:53.853268 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:53.853193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cb9xj" event={"ID":"f56cb8fc-c371-4fc0-998a-843ff6237672","Type":"ContainerStarted","Data":"8a32eb2d9be7c444480540688efe5143aae8800b1ee7aa5c24973a2c72841ec1"}
Jul 23 21:52:53.856488 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:53.856459    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4tk2" event={"ID":"9d885894-adcb-4801-aabc-8618d6acb004","Type":"ContainerStarted","Data":"97d64d570e7c739f058e0402a5386f3027883ed3c79cc52b1c407385bb7ce2c4"}
Jul 23 21:52:53.856769 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:53.856749    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-rcg2x"
Jul 23 21:52:53.875837 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:53.875774    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-rcg2x" podStartSLOduration=1.362304785 podStartE2EDuration="20.875753687s" totalImagesPullingTime="19.513448902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:53.874900971 +0000 UTC m=+96.135900744" watchObservedRunningTime="2026-07-23 21:52:53.875753687 +0000 UTC m=+96.136753415"
Jul 23 21:52:54.438742 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:54.438702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:54.522784 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:54.522744    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:54.622569 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:54.622507    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:54.866299 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:54.866209    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cb9xj" event={"ID":"f56cb8fc-c371-4fc0-998a-843ff6237672","Type":"ContainerStarted","Data":"6c4298e5832725ae9c4c85e69d8c32d22fc2129b36e9beb9e539a75d44260b7c"}
Jul 23 21:52:54.959545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:54.959509    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74f864dbdd-2s7xr"]
Jul 23 21:52:54.971544 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:54.971446    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:55.074336 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:55.074228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:55.174867 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:55.174824    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:55.240197 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:55.240104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:55.873584 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:55.873544    2575 generic.go:356] "Generic (PLEG): container finished" podID="f56cb8fc-c371-4fc0-998a-843ff6237672" containerID="6c4298e5832725ae9c4c85e69d8c32d22fc2129b36e9beb9e539a75d44260b7c" exitCode=0
Jul 23 21:52:55.874092 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:55.873781    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cb9xj" event={"ID":"f56cb8fc-c371-4fc0-998a-843ff6237672","Type":"ContainerDied","Data":"6c4298e5832725ae9c4c85e69d8c32d22fc2129b36e9beb9e539a75d44260b7c"}
Jul 23 21:52:56.147863 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:56.147728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:56.235285 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:56.235223    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:56.334791 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:56.334751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:56.436439 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:56.436348    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:56.538268 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:56.538219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:56.721243 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:56.721154    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tq988"
Jul 23 21:52:57.832106 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:57.832068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:58.890774 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:58.890703    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cb9xj" event={"ID":"f56cb8fc-c371-4fc0-998a-843ff6237672","Type":"ContainerStarted","Data":"4dc3b54b5e8cab31d5ff6ac64d268e178d5ff73179c4dc7127599dd76b719777"}
Jul 23 21:52:58.891339 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:58.890834    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cb9xj" event={"ID":"f56cb8fc-c371-4fc0-998a-843ff6237672","Type":"ContainerStarted","Data":"6f51c009a9723d3d0d7d8f510d941f2849c8381980def6509655c0b651906608"}
Jul 23 21:52:58.892848 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:58.892819    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4tk2" event={"ID":"9d885894-adcb-4801-aabc-8618d6acb004","Type":"ContainerStarted","Data":"25d324869452ebae5c2d6fe8d5587c27f694e72bbd49ef6228562ec2dd5fa13e"}
Jul 23 21:52:58.912199 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:58.912149    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cb9xj" podStartSLOduration=6.140777143 podStartE2EDuration="7.912132268s" totalImagesPullingTime="1.667917295s" totalInitContainerRuntime="103.43783ms" isStatefulPod=true podCreationTimestamp="2026-07-23 21:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:58.910250903 +0000 UTC m=+101.171250650" watchObservedRunningTime="2026-07-23 21:52:58.912132268 +0000 UTC m=+101.173131996"
Jul 23 21:52:59.733664 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:59.733577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:52:59.823983 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:52:59.823828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:01.249805 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:01.249767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:01.337770 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:01.337734    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:02.030389 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.030350    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67dfc49ccc-92xkq"]
Jul 23 21:53:02.178673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.178609    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67dfc49ccc-92xkq"]
Jul 23 21:53:02.178864 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.178779    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.181477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.181380    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 21:53:02.181704 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.181540    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 21:53:02.181704 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.181671    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 21:53:02.182144 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.182119    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5ff74\""
Jul 23 21:53:02.182760 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.182537    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 21:53:02.183135 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.182987    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 21:53:02.190223 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.189578    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 21:53:02.194518 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.194487    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:02.263224 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263151    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-service-ca\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.263224 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dhjj\" (UniqueName: \"kubernetes.io/projected/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-kube-api-access-2dhjj\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.263750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263274    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-serving-cert\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.263750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263383    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-config\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.263750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-oauth-serving-cert\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.263750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263480    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-oauth-config\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.263750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.263517    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-trusted-ca-bundle\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.290283 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.290190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:02.364228 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-service-ca\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.364228 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364226    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dhjj\" (UniqueName: \"kubernetes.io/projected/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-kube-api-access-2dhjj\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.364504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364265    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-serving-cert\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.364504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-config\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.364504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-oauth-serving-cert\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.364504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-oauth-config\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.364504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.364463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-trusted-ca-bundle\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.365669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.365336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-service-ca\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.365669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.365410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-trusted-ca-bundle\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.365669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.365580    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-config\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.365933 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.365905    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-oauth-serving-cert\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.368589 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.368538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-serving-cert\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.368718 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.368589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-oauth-config\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.375277 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.375253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dhjj\" (UniqueName: \"kubernetes.io/projected/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-kube-api-access-2dhjj\") pod \"console-67dfc49ccc-92xkq\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") " pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.495928 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.495885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:02.816424 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.816398    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67dfc49ccc-92xkq"]
Jul 23 21:53:02.819154 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:53:02.819112    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69e99b3_3af9_4a06_88c1_f1bc79dd3acb.slice/crio-5c6cf49b298d9289f74d302c0828126f5add7d88376aa463d7a6e4d5bf8277fc WatchSource:0}: Error finding container 5c6cf49b298d9289f74d302c0828126f5add7d88376aa463d7a6e4d5bf8277fc: Status 404 returned error can't find the container with id 5c6cf49b298d9289f74d302c0828126f5add7d88376aa463d7a6e4d5bf8277fc
Jul 23 21:53:02.912108 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:02.912070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfc49ccc-92xkq" event={"ID":"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb","Type":"ContainerStarted","Data":"5c6cf49b298d9289f74d302c0828126f5add7d88376aa463d7a6e4d5bf8277fc"}
Jul 23 21:53:03.918802 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:03.918758    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4tk2" event={"ID":"9d885894-adcb-4801-aabc-8618d6acb004","Type":"ContainerStarted","Data":"5d8f145d88e2ae77319f9bbafd3712f08e6bb6896772a0eca7f9c9764225c597"}
Jul 23 21:53:03.945043 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:03.944975    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d4tk2" podStartSLOduration=33.961925902 podStartE2EDuration="56.944954707s" totalImagesPullingTime="22.983028805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:52:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:03.943519885 +0000 UTC m=+106.204519615" watchObservedRunningTime="2026-07-23 21:53:03.944954707 +0000 UTC m=+106.205954439"
Jul 23 21:53:04.167863 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:04.167820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:04.266697 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:04.266545    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:06.933496 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:06.933457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfc49ccc-92xkq" event={"ID":"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb","Type":"ContainerStarted","Data":"64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b"}
Jul 23 21:53:06.952828 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:06.952773    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67dfc49ccc-92xkq" podStartSLOduration=1.764739589 podStartE2EDuration="4.952754746s" totalImagesPullingTime="3.188015157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:06.951515899 +0000 UTC m=+109.212515627" watchObservedRunningTime="2026-07-23 21:53:06.952754746 +0000 UTC m=+109.213754474"
Jul 23 21:53:08.196911 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:08.196872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:08.294113 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:08.294069    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:08.821336 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:08.821298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:08.918906 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:08.918863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:09.314679 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:09.314624    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:09.384787 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:09.384754    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:12.496470 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:12.496437    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:12.496933 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:12.496499    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:12.500882 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:12.500855    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:12.962360 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:12.962332    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:53:13.177100 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:13.177064    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:13.301171 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:13.301074    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:13.402077 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:13.402032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:16.199401 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:16.199365    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:16.256203 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:16.256167    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:19.981919 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:19.981887    2575 generic.go:356] "Generic (PLEG): container finished" podID="b22450ac-7815-41e8-954d-d065f8a75de5" containerID="5ac12461f948467b026bba3271f9003206285772369798fabb387549afe4f77d" exitCode=0
Jul 23 21:53:19.982326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:19.981964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg" event={"ID":"b22450ac-7815-41e8-954d-d065f8a75de5","Type":"ContainerDied","Data":"5ac12461f948467b026bba3271f9003206285772369798fabb387549afe4f77d"}
Jul 23 21:53:19.982326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:19.982298    2575 scope.go:117] "RemoveContainer" containerID="5ac12461f948467b026bba3271f9003206285772369798fabb387549afe4f77d"
Jul 23 21:53:19.989920 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:19.989891    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" podUID="34152caf-86bd-4ff2-8f02-7ea46621d1af" containerName="registry" containerID="cri-o://5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e" gracePeriod=30
Jul 23 21:53:20.257854 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.257790    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:53:20.427346 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427304    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-installation-pull-secrets\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427346 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427344    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kswpd\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-kube-api-access-kswpd\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427371    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427399    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-image-registry-private-configuration\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427459    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-certificates\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427499    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-bound-sa-token\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427556    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34152caf-86bd-4ff2-8f02-7ea46621d1af-ca-trust-extracted\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427669 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427584    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-trusted-ca\") pod \"34152caf-86bd-4ff2-8f02-7ea46621d1af\" (UID: \"34152caf-86bd-4ff2-8f02-7ea46621d1af\") "
Jul 23 21:53:20.427956 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.427932    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:53:20.428466 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.428441    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:53:20.430300 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.430271    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:53:20.431683 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.430616    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:53:20.431683 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.430689    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:53:20.431683 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.430723    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-kube-api-access-kswpd" (OuterVolumeSpecName: "kube-api-access-kswpd") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "kube-api-access-kswpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:53:20.432023 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.431993    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:53:20.441593 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.441567    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34152caf-86bd-4ff2-8f02-7ea46621d1af-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "34152caf-86bd-4ff2-8f02-7ea46621d1af" (UID: "34152caf-86bd-4ff2-8f02-7ea46621d1af"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529279    2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-certificates\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529311    2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-bound-sa-token\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529324    2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34152caf-86bd-4ff2-8f02-7ea46621d1af-ca-trust-extracted\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529336    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34152caf-86bd-4ff2-8f02-7ea46621d1af-trusted-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529348    2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-installation-pull-secrets\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529361    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kswpd\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-kube-api-access-kswpd\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529369 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529373    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34152caf-86bd-4ff2-8f02-7ea46621d1af-registry-tls\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.529713 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.529385    2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34152caf-86bd-4ff2-8f02-7ea46621d1af-image-registry-private-configuration\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:53:20.986726 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.986679    2575 generic.go:356] "Generic (PLEG): container finished" podID="34152caf-86bd-4ff2-8f02-7ea46621d1af" containerID="5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e" exitCode=0
Jul 23 21:53:20.987281 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.986739    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr"
Jul 23 21:53:20.987281 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.986765    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" event={"ID":"34152caf-86bd-4ff2-8f02-7ea46621d1af","Type":"ContainerDied","Data":"5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e"}
Jul 23 21:53:20.987281 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.986878    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f864dbdd-2s7xr" event={"ID":"34152caf-86bd-4ff2-8f02-7ea46621d1af","Type":"ContainerDied","Data":"67e94496b7a25325d7eb3b41fcd2e2de345c424e22df215d6c2f3ae26b4e9caf"}
Jul 23 21:53:20.987281 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.986912    2575 scope.go:117] "RemoveContainer" containerID="5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e"
Jul 23 21:53:20.988868 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.988844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-95fgg" event={"ID":"b22450ac-7815-41e8-954d-d065f8a75de5","Type":"ContainerStarted","Data":"d353292866a5c7fe4f0b41f6c5adef6b9612a0ec255ca092fbad5c4ebdc4a4eb"}
Jul 23 21:53:20.995781 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.995763    2575 scope.go:117] "RemoveContainer" containerID="5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e"
Jul 23 21:53:20.996053 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:53:20.996032    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e\": container with ID starting with 5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e not found: ID does not exist" containerID="5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e"
Jul 23 21:53:20.996114 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:20.996067    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e"} err="failed to get container status \"5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e\": rpc error: code = NotFound desc = could not find container \"5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e\": container with ID starting with 5c248c3407e9b6d927be368fa40b81a10dee338e51bf446b8cdb1e450018e85e not found: ID does not exist"
Jul 23 21:53:21.022828 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:21.022798    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74f864dbdd-2s7xr"]
Jul 23 21:53:21.028326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:21.028303    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74f864dbdd-2s7xr"]
Jul 23 21:53:22.226262 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:22.226229    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34152caf-86bd-4ff2-8f02-7ea46621d1af" path="/var/lib/kubelet/pods/34152caf-86bd-4ff2-8f02-7ea46621d1af/volumes"
Jul 23 21:53:24.471741 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:24.471708    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:24.572716 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:24.572674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:24.673091 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:24.673052    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:28.195007 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:28.194968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:53:28.197305 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:28.197272    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6571af9-eab4-4e45-8560-9c0065dcabfd-metrics-certs\") pod \"network-metrics-daemon-9t6qd\" (UID: \"f6571af9-eab4-4e45-8560-9c0065dcabfd\") " pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:53:28.348234 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:28.348201    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jd2p5\""
Jul 23 21:53:28.356058 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:28.356025    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9t6qd"
Jul 23 21:53:28.574040 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:28.573969    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9t6qd"]
Jul 23 21:53:28.576987 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:53:28.576962    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6571af9_eab4_4e45_8560_9c0065dcabfd.slice/crio-f151bc32fab58e73fa5a72295b7474bef00dbe76a0c8d1fb6b99bf2ca3ab86f0 WatchSource:0}: Error finding container f151bc32fab58e73fa5a72295b7474bef00dbe76a0c8d1fb6b99bf2ca3ab86f0: Status 404 returned error can't find the container with id f151bc32fab58e73fa5a72295b7474bef00dbe76a0c8d1fb6b99bf2ca3ab86f0
Jul 23 21:53:29.020585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:29.020542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9t6qd" event={"ID":"f6571af9-eab4-4e45-8560-9c0065dcabfd","Type":"ContainerStarted","Data":"f151bc32fab58e73fa5a72295b7474bef00dbe76a0c8d1fb6b99bf2ca3ab86f0"}
Jul 23 21:53:31.029252 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.029207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9t6qd" event={"ID":"f6571af9-eab4-4e45-8560-9c0065dcabfd","Type":"ContainerStarted","Data":"8529c7994fea76b948cdadb3f0e0be9a34b39d18df5e71672e94bc8e673eb4db"}
Jul 23 21:53:31.029737 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.029306    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9t6qd" event={"ID":"f6571af9-eab4-4e45-8560-9c0065dcabfd","Type":"ContainerStarted","Data":"deded354495f045ed821e72ae59f9627326af66de7431324415f0771df21e436"}
Jul 23 21:53:31.030626 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.030599    2575 generic.go:356] "Generic (PLEG): container finished" podID="11a9d26c-98ae-41b9-bd98-e06274f9d004" containerID="a03f1d9baa36e113e515b1241988a9ae34390e5c5f43556ceec2e85f2811df11" exitCode=0
Jul 23 21:53:31.030718 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.030665    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n" event={"ID":"11a9d26c-98ae-41b9-bd98-e06274f9d004","Type":"ContainerDied","Data":"a03f1d9baa36e113e515b1241988a9ae34390e5c5f43556ceec2e85f2811df11"}
Jul 23 21:53:31.030921 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.030908    2575 scope.go:117] "RemoveContainer" containerID="a03f1d9baa36e113e515b1241988a9ae34390e5c5f43556ceec2e85f2811df11"
Jul 23 21:53:31.048443 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.048396    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9t6qd" podStartSLOduration=130.710438557 podStartE2EDuration="2m12.048381397s" totalImagesPullingTime="1.33794284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:31.046517962 +0000 UTC m=+133.307517716" watchObservedRunningTime="2026-07-23 21:53:31.048381397 +0000 UTC m=+133.309381125"
Jul 23 21:53:31.277646 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.277598    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:31.379442 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:31.379346    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:32.036587 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:32.036551    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hdc7n" event={"ID":"11a9d26c-98ae-41b9-bd98-e06274f9d004","Type":"ContainerStarted","Data":"52c5cb67fe5b98ebba6c0fc16ac318dbb5d6237a3694c31f0aa6db9b9044d034"}
Jul 23 21:53:36.943288 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:36.943251    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:37.042686 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:37.042622    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:37.823786 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:37.823753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:37.880347 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:37.880310    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:42.079856 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:42.079816    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea1e6f27-49dc-457b-871f-699f016f9bdf" containerID="803bd05dba5086e529f9343476f36768a1530246418377511dbdba1336ab2487" exitCode=0
Jul 23 21:53:42.080335 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:42.079878    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-4k9jn" event={"ID":"ea1e6f27-49dc-457b-871f-699f016f9bdf","Type":"ContainerDied","Data":"803bd05dba5086e529f9343476f36768a1530246418377511dbdba1336ab2487"}
Jul 23 21:53:42.080335 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:42.080308    2575 scope.go:117] "RemoveContainer" containerID="803bd05dba5086e529f9343476f36768a1530246418377511dbdba1336ab2487"
Jul 23 21:53:43.085206 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:43.085170    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-4k9jn" event={"ID":"ea1e6f27-49dc-457b-871f-699f016f9bdf","Type":"ContainerStarted","Data":"613af09ae614975cb86ad9fae2cde9fd34e7696273cd2b5b0bcccef66b304ea4"}
Jul 23 21:53:44.043231 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:44.043196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:44.144946 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:44.144909    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:50.333226 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:50.333187    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:50.396297 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:50.396257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:53:56.395922 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:53:56.395858    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:01.217442 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.217397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:01.274526 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.274492    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bcc799d49-tgdq4"]
Jul 23 21:54:01.274914 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.274894    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34152caf-86bd-4ff2-8f02-7ea46621d1af" containerName="registry"
Jul 23 21:54:01.274914 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.274916    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="34152caf-86bd-4ff2-8f02-7ea46621d1af" containerName="registry"
Jul 23 21:54:01.275088 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.275004    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="34152caf-86bd-4ff2-8f02-7ea46621d1af" containerName="registry"
Jul 23 21:54:01.277524 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.277497    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.291072 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.291036    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bcc799d49-tgdq4"]
Jul 23 21:54:01.314169 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.314142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:01.351977 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.351946    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-oauth-config\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.352138 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.352001    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-oauth-serving-cert\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.352138 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.352083    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-service-ca\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.352138 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.352128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-serving-cert\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.352238 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.352160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-trusted-ca-bundle\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.352238 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.352216    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-console-config\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.352299 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.352239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5sld\" (UniqueName: \"kubernetes.io/projected/93ab960f-3f43-4966-a336-043f98b88d9f-kube-api-access-z5sld\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.401625 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.401585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:01.453619 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453580    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-oauth-serving-cert\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.453815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-service-ca\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.453815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-serving-cert\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.453815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-trusted-ca-bundle\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.453815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-console-config\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.453815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453747    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5sld\" (UniqueName: \"kubernetes.io/projected/93ab960f-3f43-4966-a336-043f98b88d9f-kube-api-access-z5sld\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.453815 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.453788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-oauth-config\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.454471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.454443    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-service-ca\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.454585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.454466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-oauth-serving-cert\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.454585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.454550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-console-config\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.454585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.454566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-trusted-ca-bundle\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.456299 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.456264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-oauth-config\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.456387 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.456314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-serving-cert\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.462724 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.462692    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5sld\" (UniqueName: \"kubernetes.io/projected/93ab960f-3f43-4966-a336-043f98b88d9f-kube-api-access-z5sld\") pod \"console-5bcc799d49-tgdq4\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") " pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.500158 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.500079    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:01.586225 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.586190    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:01.718339 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:01.718315    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bcc799d49-tgdq4"]
Jul 23 21:54:01.720866 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:54:01.720836    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ab960f_3f43_4966_a336_043f98b88d9f.slice/crio-b0cd7c802c71ef6930ba7af39d5fbdbd4e69ced83a464ac1c0bccab337c7f3ba WatchSource:0}: Error finding container b0cd7c802c71ef6930ba7af39d5fbdbd4e69ced83a464ac1c0bccab337c7f3ba: Status 404 returned error can't find the container with id b0cd7c802c71ef6930ba7af39d5fbdbd4e69ced83a464ac1c0bccab337c7f3ba
Jul 23 21:54:02.066041 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:02.065950    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:02.159032 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:02.158999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc799d49-tgdq4" event={"ID":"93ab960f-3f43-4966-a336-043f98b88d9f","Type":"ContainerStarted","Data":"aa28ca03033b955a13b9a3479e99112f366c93399c9926bb2beeef756766ba7e"}
Jul 23 21:54:02.159233 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:02.159055    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc799d49-tgdq4" event={"ID":"93ab960f-3f43-4966-a336-043f98b88d9f","Type":"ContainerStarted","Data":"b0cd7c802c71ef6930ba7af39d5fbdbd4e69ced83a464ac1c0bccab337c7f3ba"}
Jul 23 21:54:02.180109 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:02.180043    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bcc799d49-tgdq4" podStartSLOduration=1.1800225229999999 podStartE2EDuration="1.180022523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:54:02.179179985 +0000 UTC m=+164.440179753" watchObservedRunningTime="2026-07-23 21:54:02.180022523 +0000 UTC m=+164.441022252"
Jul 23 21:54:11.587222 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:11.587139    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:11.587222 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:11.587202    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:11.592072 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:11.592044    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:12.202293 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:12.202267    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:54:12.259795 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:12.259758    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67dfc49ccc-92xkq"]
Jul 23 21:54:12.352245 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:12.352208    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:12.418956 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:12.418924    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:12.484281 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:12.484195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:37.281458 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.281418    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67dfc49ccc-92xkq" podUID="b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" containerName="console" containerID="cri-o://64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b" gracePeriod=15
Jul 23 21:54:37.377393 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.377362    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:37.437266 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.437232    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:37.538128 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.538067    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:54:37.559621 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559586    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-serving-cert\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.559803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559667    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-trusted-ca-bundle\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.559803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559695    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-oauth-serving-cert\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.559803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559724    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dhjj\" (UniqueName: \"kubernetes.io/projected/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-kube-api-access-2dhjj\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.559803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559765    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-oauth-config\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.559803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559793    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-service-ca\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.560047 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.559852    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-config\") pod \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\" (UID: \"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb\") "
Jul 23 21:54:37.560443 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.560277    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:54:37.560443 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.560299    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-service-ca" (OuterVolumeSpecName: "service-ca") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:54:37.560619 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.560599    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.560689 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.560620    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-oauth-serving-cert\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.560735 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.560684    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:54:37.560927 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.560905    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-config" (OuterVolumeSpecName: "console-config") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:54:37.562426 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.562383    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:54:37.562554 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.562521    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-kube-api-access-2dhjj" (OuterVolumeSpecName: "kube-api-access-2dhjj") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "kube-api-access-2dhjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:54:37.562613 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.562553    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" (UID: "b69e99b3-3af9-4a06-88c1-f1bc79dd3acb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:54:37.660974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.660932    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-config\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.660974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.660968    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-serving-cert\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.660974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.660980    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-trusted-ca-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.661193 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.660992    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dhjj\" (UniqueName: \"kubernetes.io/projected/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-kube-api-access-2dhjj\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.661193 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.661004    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb-console-oauth-config\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:54:37.680224 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:37.680186    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:38.295052 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.295011    2575 generic.go:356] "Generic (PLEG): container finished" podID="b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" containerID="64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b" exitCode=0
Jul 23 21:54:38.295052 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.295052    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfc49ccc-92xkq"
Jul 23 21:54:38.295539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.295093    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfc49ccc-92xkq" event={"ID":"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb","Type":"ContainerDied","Data":"64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b"}
Jul 23 21:54:38.295539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.295145    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfc49ccc-92xkq" event={"ID":"b69e99b3-3af9-4a06-88c1-f1bc79dd3acb","Type":"ContainerDied","Data":"5c6cf49b298d9289f74d302c0828126f5add7d88376aa463d7a6e4d5bf8277fc"}
Jul 23 21:54:38.295539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.295167    2575 scope.go:117] "RemoveContainer" containerID="64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b"
Jul 23 21:54:38.303273 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.303252    2575 scope.go:117] "RemoveContainer" containerID="64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b"
Jul 23 21:54:38.303519 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:54:38.303493    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b\": container with ID starting with 64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b not found: ID does not exist" containerID="64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b"
Jul 23 21:54:38.303580 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.303523    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b"} err="failed to get container status \"64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b\": rpc error: code = NotFound desc = could not find container \"64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b\": container with ID starting with 64fd76c679e9a4a4c454eb499be835db56e64762be05f0fcef71f681d043b82b not found: ID does not exist"
Jul 23 21:54:38.311901 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.311879    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67dfc49ccc-92xkq"]
Jul 23 21:54:38.315233 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:38.315211    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67dfc49ccc-92xkq"]
Jul 23 21:54:40.225962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:40.225926    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" path="/var/lib/kubelet/pods/b69e99b3-3af9-4a06-88c1-f1bc79dd3acb/volumes"
Jul 23 21:54:45.154376 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:45.154339    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:46.657506 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:46.657463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:46.753622 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:46.753581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:54:47.411820 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.411780    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"]
Jul 23 21:54:47.412085 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.412073    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" containerName="console"
Jul 23 21:54:47.412128 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.412087    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" containerName="console"
Jul 23 21:54:47.412169 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.412136    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b69e99b3-3af9-4a06-88c1-f1bc79dd3acb" containerName="console"
Jul 23 21:54:47.414980 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.414963    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.417565 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.417534    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 21:54:47.417565 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.417565    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v4wk5\""
Jul 23 21:54:47.417780 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.417615    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 21:54:47.425305 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.425281    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"]
Jul 23 21:54:47.430905 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.430880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7glm2\" (UniqueName: \"kubernetes.io/projected/96e4c5df-58a6-4797-9444-d6e99e468840-kube-api-access-7glm2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.431004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.430914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.431004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.430985    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.531514 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.531480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7glm2\" (UniqueName: \"kubernetes.io/projected/96e4c5df-58a6-4797-9444-d6e99e468840-kube-api-access-7glm2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.531514 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.531517    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.531744 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.531559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.531946 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.531929    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.532002 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.531987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.541501 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.541476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7glm2\" (UniqueName: \"kubernetes.io/projected/96e4c5df-58a6-4797-9444-d6e99e468840-kube-api-access-7glm2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.723946 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.723868    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:54:47.851982 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:47.851889    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"]
Jul 23 21:54:47.854876 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:54:47.854842    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96e4c5df_58a6_4797_9444_d6e99e468840.slice/crio-2c7ae98ccfb6fab76b10b7698476f9a60d7d10b7fee219b0d1fbd07c89917c74 WatchSource:0}: Error finding container 2c7ae98ccfb6fab76b10b7698476f9a60d7d10b7fee219b0d1fbd07c89917c74: Status 404 returned error can't find the container with id 2c7ae98ccfb6fab76b10b7698476f9a60d7d10b7fee219b0d1fbd07c89917c74
Jul 23 21:54:48.337997 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:48.337954    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf" event={"ID":"96e4c5df-58a6-4797-9444-d6e99e468840","Type":"ContainerStarted","Data":"2c7ae98ccfb6fab76b10b7698476f9a60d7d10b7fee219b0d1fbd07c89917c74"}
Jul 23 21:54:54.365732 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:54.365691    2575 generic.go:356] "Generic (PLEG): container finished" podID="96e4c5df-58a6-4797-9444-d6e99e468840" containerID="1617358d1296a782944650d566ff73eb7e7342a8a5f1d4e62c1660de28ffd4a5" exitCode=0
Jul 23 21:54:54.366145 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:54.365747    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf" event={"ID":"96e4c5df-58a6-4797-9444-d6e99e468840","Type":"ContainerDied","Data":"1617358d1296a782944650d566ff73eb7e7342a8a5f1d4e62c1660de28ffd4a5"}
Jul 23 21:54:57.380263 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:57.380222    2575 generic.go:356] "Generic (PLEG): container finished" podID="96e4c5df-58a6-4797-9444-d6e99e468840" containerID="9abe325385c3d9460c23d9197855b50082a3e5311ef155014896b84e3fcae3c2" exitCode=0
Jul 23 21:54:57.380626 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:54:57.380283    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf" event={"ID":"96e4c5df-58a6-4797-9444-d6e99e468840","Type":"ContainerDied","Data":"9abe325385c3d9460c23d9197855b50082a3e5311ef155014896b84e3fcae3c2"}
Jul 23 21:55:04.415677 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:04.415623    2575 generic.go:356] "Generic (PLEG): container finished" podID="96e4c5df-58a6-4797-9444-d6e99e468840" containerID="80d8c97d1961aa133db0aac2f40bef830f931ad25cf92fc9750b9df2f0ea0b4d" exitCode=0
Jul 23 21:55:04.416038 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:04.415728    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf" event={"ID":"96e4c5df-58a6-4797-9444-d6e99e468840","Type":"ContainerDied","Data":"80d8c97d1961aa133db0aac2f40bef830f931ad25cf92fc9750b9df2f0ea0b4d"}
Jul 23 21:55:05.545731 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.545702    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:55:05.585595 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.585565    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-bundle\") pod \"96e4c5df-58a6-4797-9444-d6e99e468840\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") "
Jul 23 21:55:05.585752 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.585615    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-util\") pod \"96e4c5df-58a6-4797-9444-d6e99e468840\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") "
Jul 23 21:55:05.585752 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.585688    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7glm2\" (UniqueName: \"kubernetes.io/projected/96e4c5df-58a6-4797-9444-d6e99e468840-kube-api-access-7glm2\") pod \"96e4c5df-58a6-4797-9444-d6e99e468840\" (UID: \"96e4c5df-58a6-4797-9444-d6e99e468840\") "
Jul 23 21:55:05.586193 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.586172    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-bundle" (OuterVolumeSpecName: "bundle") pod "96e4c5df-58a6-4797-9444-d6e99e468840" (UID: "96e4c5df-58a6-4797-9444-d6e99e468840"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:55:05.587961 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.587934    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e4c5df-58a6-4797-9444-d6e99e468840-kube-api-access-7glm2" (OuterVolumeSpecName: "kube-api-access-7glm2") pod "96e4c5df-58a6-4797-9444-d6e99e468840" (UID: "96e4c5df-58a6-4797-9444-d6e99e468840"). InnerVolumeSpecName "kube-api-access-7glm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:55:05.589915 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.589890    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-util" (OuterVolumeSpecName: "util") pod "96e4c5df-58a6-4797-9444-d6e99e468840" (UID: "96e4c5df-58a6-4797-9444-d6e99e468840"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:55:05.686609 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.686523    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7glm2\" (UniqueName: \"kubernetes.io/projected/96e4c5df-58a6-4797-9444-d6e99e468840-kube-api-access-7glm2\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:05.686609 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.686555    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:05.686609 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:05.686568    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96e4c5df-58a6-4797-9444-d6e99e468840-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:06.426103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:06.426068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf" event={"ID":"96e4c5df-58a6-4797-9444-d6e99e468840","Type":"ContainerDied","Data":"2c7ae98ccfb6fab76b10b7698476f9a60d7d10b7fee219b0d1fbd07c89917c74"}
Jul 23 21:55:06.426289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:06.426107    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dn4pf"
Jul 23 21:55:06.426289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:06.426118    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c7ae98ccfb6fab76b10b7698476f9a60d7d10b7fee219b0d1fbd07c89917c74"
Jul 23 21:55:08.459676 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:08.459607    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:09.345102 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:09.345060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:10.435624 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.435576    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"]
Jul 23 21:55:10.436100 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436081    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="pull"
Jul 23 21:55:10.436164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436101    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="pull"
Jul 23 21:55:10.436164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436113    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="extract"
Jul 23 21:55:10.436164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436122    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="extract"
Jul 23 21:55:10.436164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436133    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="util"
Jul 23 21:55:10.436164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436142    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="util"
Jul 23 21:55:10.436388 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.436218    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e4c5df-58a6-4797-9444-d6e99e468840" containerName="extract"
Jul 23 21:55:10.438148 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.438126    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.440514 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.440488    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 23 21:55:10.440673 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.440532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l8j6r\""
Jul 23 21:55:10.447475 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.447451    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 23 21:55:10.456594 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.456556    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"]
Jul 23 21:55:10.523944 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.523899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4h94\" (UniqueName: \"kubernetes.io/projected/5df15e97-438a-4fcf-8ae5-95506756317e-kube-api-access-w4h94\") pod \"cert-manager-operator-controller-manager-55c444bf86-smskd\" (UID: \"5df15e97-438a-4fcf-8ae5-95506756317e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.524105 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.523959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5df15e97-438a-4fcf-8ae5-95506756317e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-smskd\" (UID: \"5df15e97-438a-4fcf-8ae5-95506756317e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.625342 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.625294    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4h94\" (UniqueName: \"kubernetes.io/projected/5df15e97-438a-4fcf-8ae5-95506756317e-kube-api-access-w4h94\") pod \"cert-manager-operator-controller-manager-55c444bf86-smskd\" (UID: \"5df15e97-438a-4fcf-8ae5-95506756317e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.625521 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.625433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5df15e97-438a-4fcf-8ae5-95506756317e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-smskd\" (UID: \"5df15e97-438a-4fcf-8ae5-95506756317e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.625796 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.625781    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5df15e97-438a-4fcf-8ae5-95506756317e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-smskd\" (UID: \"5df15e97-438a-4fcf-8ae5-95506756317e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.635677 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.635648    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4h94\" (UniqueName: \"kubernetes.io/projected/5df15e97-438a-4fcf-8ae5-95506756317e-kube-api-access-w4h94\") pod \"cert-manager-operator-controller-manager-55c444bf86-smskd\" (UID: \"5df15e97-438a-4fcf-8ae5-95506756317e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.747824 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.747740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"
Jul 23 21:55:10.901011 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:10.900982    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd"]
Jul 23 21:55:10.903867 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:55:10.903837    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df15e97_438a_4fcf_8ae5_95506756317e.slice/crio-36f90e05136cfdbe37aadd738bdb785b5c4fdd740305ff2fcdd61196211423c8 WatchSource:0}: Error finding container 36f90e05136cfdbe37aadd738bdb785b5c4fdd740305ff2fcdd61196211423c8: Status 404 returned error can't find the container with id 36f90e05136cfdbe37aadd738bdb785b5c4fdd740305ff2fcdd61196211423c8
Jul 23 21:55:11.447100 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:11.447061    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd" event={"ID":"5df15e97-438a-4fcf-8ae5-95506756317e","Type":"ContainerStarted","Data":"36f90e05136cfdbe37aadd738bdb785b5c4fdd740305ff2fcdd61196211423c8"}
Jul 23 21:55:12.292533 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:12.292489    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:12.376397 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:12.376357    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:13.457378 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:13.457340    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd" event={"ID":"5df15e97-438a-4fcf-8ae5-95506756317e","Type":"ContainerStarted","Data":"a55c0ee59c8088759ea9fbfafea3249ef53ad3db1c320c786d352c5ae4fce1d4"}
Jul 23 21:55:13.483200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:13.480901    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smskd" podStartSLOduration=1.860631677 podStartE2EDuration="3.480880834s" totalImagesPullingTime="1.620249157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:55:13.477436967 +0000 UTC m=+235.738436692" watchObservedRunningTime="2026-07-23 21:55:13.480880834 +0000 UTC m=+235.741880562"
Jul 23 21:55:13.986010 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:13.985956    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:14.085780 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.085740    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:14.310601 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.310521    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"]
Jul 23 21:55:14.313008 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.312990    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.315736 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.315712    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 21:55:14.319501 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.319481    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 21:55:14.319698 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.319680    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v4wk5\""
Jul 23 21:55:14.336803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.336782    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"]
Jul 23 21:55:14.455280 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.455241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.455477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.455286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmf5h\" (UniqueName: \"kubernetes.io/projected/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-kube-api-access-mmf5h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.455477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.455318    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.504734 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.504699    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:14.556009 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.555960    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.556009 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.556013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmf5h\" (UniqueName: \"kubernetes.io/projected/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-kube-api-access-mmf5h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.556273 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.556137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.556439 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.556408    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.556439 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.556432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.564759 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.564699    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmf5h\" (UniqueName: \"kubernetes.io/projected/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-kube-api-access-mmf5h\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.595432 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.595389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:14.622390 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.622354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:14.755808 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.755781    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"]
Jul 23 21:55:14.758157 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:55:14.758126    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbf2a1b_d875_4d86_9be7_2bdbf644a974.slice/crio-5781cf76beff57647cc09a5d37ce61aec78aa8b426b651725ee33618a83bfed1 WatchSource:0}: Error finding container 5781cf76beff57647cc09a5d37ce61aec78aa8b426b651725ee33618a83bfed1: Status 404 returned error can't find the container with id 5781cf76beff57647cc09a5d37ce61aec78aa8b426b651725ee33618a83bfed1
Jul 23 21:55:14.792991 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.792964    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-svzbs"]
Jul 23 21:55:14.794947 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.794931    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:14.797214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.797193    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 21:55:14.797417 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.797400    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bwj4z\""
Jul 23 21:55:14.797602 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.797583    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 21:55:14.805698 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.805678    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-svzbs"]
Jul 23 21:55:14.858948 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.858923    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-svzbs\" (UID: \"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:14.859059 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.859035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5pgf\" (UniqueName: \"kubernetes.io/projected/2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1-kube-api-access-b5pgf\") pod \"cert-manager-webhook-6486ffd997-svzbs\" (UID: \"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:14.960211 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.960163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-svzbs\" (UID: \"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:14.960392 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.960243    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5pgf\" (UniqueName: \"kubernetes.io/projected/2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1-kube-api-access-b5pgf\") pod \"cert-manager-webhook-6486ffd997-svzbs\" (UID: \"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:14.969843 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.969818    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-svzbs\" (UID: \"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:14.969953 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:14.969910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5pgf\" (UniqueName: \"kubernetes.io/projected/2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1-kube-api-access-b5pgf\") pod \"cert-manager-webhook-6486ffd997-svzbs\" (UID: \"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:15.125476 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:15.125444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:15.275123 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:15.275084    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-svzbs"]
Jul 23 21:55:15.280927 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:55:15.280900    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dd9019c_cd47_43c9_a4eb_0fd5ecec1bc1.slice/crio-8443f00d6e1aeb6bd118677eb2119ea9d77ea0a23a41d93fc69b460ccb3e8928 WatchSource:0}: Error finding container 8443f00d6e1aeb6bd118677eb2119ea9d77ea0a23a41d93fc69b460ccb3e8928: Status 404 returned error can't find the container with id 8443f00d6e1aeb6bd118677eb2119ea9d77ea0a23a41d93fc69b460ccb3e8928
Jul 23 21:55:15.467686 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:15.467583    2575 generic.go:356] "Generic (PLEG): container finished" podID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerID="be7350156953127fe796dc128e27749615ec03e1dd75d894880b1a1d294319c9" exitCode=0
Jul 23 21:55:15.467840 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:15.467678    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7" event={"ID":"9dbf2a1b-d875-4d86-9be7-2bdbf644a974","Type":"ContainerDied","Data":"be7350156953127fe796dc128e27749615ec03e1dd75d894880b1a1d294319c9"}
Jul 23 21:55:15.467840 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:15.467738    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7" event={"ID":"9dbf2a1b-d875-4d86-9be7-2bdbf644a974","Type":"ContainerStarted","Data":"5781cf76beff57647cc09a5d37ce61aec78aa8b426b651725ee33618a83bfed1"}
Jul 23 21:55:15.468987 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:15.468961    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs" event={"ID":"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1","Type":"ContainerStarted","Data":"8443f00d6e1aeb6bd118677eb2119ea9d77ea0a23a41d93fc69b460ccb3e8928"}
Jul 23 21:55:16.711258 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:16.711213    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:16.911177 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:16.911141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:19.067487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.067431    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:19.122911 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.122854    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:19.494416 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.494382    2575 generic.go:356] "Generic (PLEG): container finished" podID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerID="5a304921157d5f29fc23e21c3d18067e42ff56282430ef16a6535b8be4c86867" exitCode=0
Jul 23 21:55:19.494585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.494471    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7" event={"ID":"9dbf2a1b-d875-4d86-9be7-2bdbf644a974","Type":"ContainerDied","Data":"5a304921157d5f29fc23e21c3d18067e42ff56282430ef16a6535b8be4c86867"}
Jul 23 21:55:19.496039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.496017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs" event={"ID":"2dd9019c-cd47-43c9-a4eb-0fd5ecec1bc1","Type":"ContainerStarted","Data":"035ca6e12ca2f3159f796bb00870920c67268245128f18187c41123ff5aeb1fd"}
Jul 23 21:55:19.496146 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.496132    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:19.527612 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:19.527566    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs" podStartSLOduration=2.27318217 podStartE2EDuration="5.52755187s" totalImagesPullingTime="3.2543697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:55:19.5268403 +0000 UTC m=+241.787840051" watchObservedRunningTime="2026-07-23 21:55:19.52755187 +0000 UTC m=+241.788551598"
Jul 23 21:55:20.502539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:20.502502    2575 generic.go:356] "Generic (PLEG): container finished" podID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerID="9a768baf7e9c37f48680493437080f77051d6e650d3d8ee247e6de1edf2f8060" exitCode=0
Jul 23 21:55:20.503008 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:20.502586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7" event={"ID":"9dbf2a1b-d875-4d86-9be7-2bdbf644a974","Type":"ContainerDied","Data":"9a768baf7e9c37f48680493437080f77051d6e650d3d8ee247e6de1edf2f8060"}
Jul 23 21:55:21.633184 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.633161    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:21.716289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.716261    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-bundle\") pod \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") "
Jul 23 21:55:21.716289 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.716292    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-util\") pod \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") "
Jul 23 21:55:21.716499 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.716342    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmf5h\" (UniqueName: \"kubernetes.io/projected/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-kube-api-access-mmf5h\") pod \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\" (UID: \"9dbf2a1b-d875-4d86-9be7-2bdbf644a974\") "
Jul 23 21:55:21.716794 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.716765    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-bundle" (OuterVolumeSpecName: "bundle") pod "9dbf2a1b-d875-4d86-9be7-2bdbf644a974" (UID: "9dbf2a1b-d875-4d86-9be7-2bdbf644a974"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:55:21.718491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.718462    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-kube-api-access-mmf5h" (OuterVolumeSpecName: "kube-api-access-mmf5h") pod "9dbf2a1b-d875-4d86-9be7-2bdbf644a974" (UID: "9dbf2a1b-d875-4d86-9be7-2bdbf644a974"). InnerVolumeSpecName "kube-api-access-mmf5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:55:21.720891 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.720870    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-util" (OuterVolumeSpecName: "util") pod "9dbf2a1b-d875-4d86-9be7-2bdbf644a974" (UID: "9dbf2a1b-d875-4d86-9be7-2bdbf644a974"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:55:21.817458 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.817385    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmf5h\" (UniqueName: \"kubernetes.io/projected/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-kube-api-access-mmf5h\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:21.817458 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.817412    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:21.817458 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:21.817421    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9dbf2a1b-d875-4d86-9be7-2bdbf644a974-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:22.317737 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:22.317698    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:22.388517 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:22.388475    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:22.512960 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:22.512872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7" event={"ID":"9dbf2a1b-d875-4d86-9be7-2bdbf644a974","Type":"ContainerDied","Data":"5781cf76beff57647cc09a5d37ce61aec78aa8b426b651725ee33618a83bfed1"}
Jul 23 21:55:22.512960 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:22.512910    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f69tk7"
Jul 23 21:55:22.512960 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:22.512948    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5781cf76beff57647cc09a5d37ce61aec78aa8b426b651725ee33618a83bfed1"
Jul 23 21:55:24.891738 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:24.891704    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:25.505440 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:25.505406    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-svzbs"
Jul 23 21:55:26.561965 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.561932    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:26.662398 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.662360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:26.742099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742066    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s2xg6"]
Jul 23 21:55:26.742358 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742347    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="extract"
Jul 23 21:55:26.742402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742360    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="extract"
Jul 23 21:55:26.742402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742375    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="util"
Jul 23 21:55:26.742402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742381    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="util"
Jul 23 21:55:26.742402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742391    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="pull"
Jul 23 21:55:26.742402 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742396    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="pull"
Jul 23 21:55:26.742547 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.742451    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dbf2a1b-d875-4d86-9be7-2bdbf644a974" containerName="extract"
Jul 23 21:55:26.746047 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.746026    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:26.748241 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.748222    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-b6hzj\""
Jul 23 21:55:26.756007 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.755983    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s2xg6"]
Jul 23 21:55:26.860251 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.860215    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a708bcc1-66ce-4c81-b03a-bcb7a341dccc-bound-sa-token\") pod \"cert-manager-55b4d954d5-s2xg6\" (UID: \"a708bcc1-66ce-4c81-b03a-bcb7a341dccc\") " pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:26.860395 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.860261    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvpcb\" (UniqueName: \"kubernetes.io/projected/a708bcc1-66ce-4c81-b03a-bcb7a341dccc-kube-api-access-xvpcb\") pod \"cert-manager-55b4d954d5-s2xg6\" (UID: \"a708bcc1-66ce-4c81-b03a-bcb7a341dccc\") " pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:26.961021 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.960984    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a708bcc1-66ce-4c81-b03a-bcb7a341dccc-bound-sa-token\") pod \"cert-manager-55b4d954d5-s2xg6\" (UID: \"a708bcc1-66ce-4c81-b03a-bcb7a341dccc\") " pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:26.961185 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.961027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvpcb\" (UniqueName: \"kubernetes.io/projected/a708bcc1-66ce-4c81-b03a-bcb7a341dccc-kube-api-access-xvpcb\") pod \"cert-manager-55b4d954d5-s2xg6\" (UID: \"a708bcc1-66ce-4c81-b03a-bcb7a341dccc\") " pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:26.968941 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.968911    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a708bcc1-66ce-4c81-b03a-bcb7a341dccc-bound-sa-token\") pod \"cert-manager-55b4d954d5-s2xg6\" (UID: \"a708bcc1-66ce-4c81-b03a-bcb7a341dccc\") " pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:26.969056 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:26.969025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvpcb\" (UniqueName: \"kubernetes.io/projected/a708bcc1-66ce-4c81-b03a-bcb7a341dccc-kube-api-access-xvpcb\") pod \"cert-manager-55b4d954d5-s2xg6\" (UID: \"a708bcc1-66ce-4c81-b03a-bcb7a341dccc\") " pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:27.055033 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:27.054998    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s2xg6"
Jul 23 21:55:27.183586 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:27.183553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s2xg6"]
Jul 23 21:55:27.186297 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:55:27.186264    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda708bcc1_66ce_4c81_b03a_bcb7a341dccc.slice/crio-796f8b83ef05af80ecfe9ae79c708c625144fc9d8c40ad7257cd7bfaba963247 WatchSource:0}: Error finding container 796f8b83ef05af80ecfe9ae79c708c625144fc9d8c40ad7257cd7bfaba963247: Status 404 returned error can't find the container with id 796f8b83ef05af80ecfe9ae79c708c625144fc9d8c40ad7257cd7bfaba963247
Jul 23 21:55:27.534896 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:27.534814    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s2xg6" event={"ID":"a708bcc1-66ce-4c81-b03a-bcb7a341dccc","Type":"ContainerStarted","Data":"fb039ca406fc2e748e98d91945949cd851ce7f6925e34e1344fbb96bf5eca189"}
Jul 23 21:55:27.534896 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:27.534855    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s2xg6" event={"ID":"a708bcc1-66ce-4c81-b03a-bcb7a341dccc","Type":"ContainerStarted","Data":"796f8b83ef05af80ecfe9ae79c708c625144fc9d8c40ad7257cd7bfaba963247"}
Jul 23 21:55:27.552483 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:27.552423    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-s2xg6" podStartSLOduration=1.552408089 podStartE2EDuration="1.552408089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:55:27.551559571 +0000 UTC m=+249.812559319" watchObservedRunningTime="2026-07-23 21:55:27.552408089 +0000 UTC m=+249.813407817"
Jul 23 21:55:32.534143 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:32.534110    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:32.635740 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:32.635701    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:34.015474 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.015436    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"]
Jul 23 21:55:34.019068 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.019050    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.021304 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.021283    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 21:55:34.021917 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.021894    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v4wk5\""
Jul 23 21:55:34.022018 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.021934    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 21:55:34.027803 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.027777    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"]
Jul 23 21:55:34.125659 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.125596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48r2v\" (UniqueName: \"kubernetes.io/projected/bbe5cad6-2bb2-488f-a91f-14a76d754789-kube-api-access-48r2v\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.125812 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.125701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.125812 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.125756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.226855 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.226821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48r2v\" (UniqueName: \"kubernetes.io/projected/bbe5cad6-2bb2-488f-a91f-14a76d754789-kube-api-access-48r2v\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.227041 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.226865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.227041 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.226894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.227229 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.227207    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.227283 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.227235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.237609 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.237579    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48r2v\" (UniqueName: \"kubernetes.io/projected/bbe5cad6-2bb2-488f-a91f-14a76d754789-kube-api-access-48r2v\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.329419 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.329348    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:34.460504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.460478    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"]
Jul 23 21:55:34.463044 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:55:34.463015    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe5cad6_2bb2_488f_a91f_14a76d754789.slice/crio-0026877c671241a11369434175d9c495b6ab4febec093c47d57340c1913c68dc WatchSource:0}: Error finding container 0026877c671241a11369434175d9c495b6ab4febec093c47d57340c1913c68dc: Status 404 returned error can't find the container with id 0026877c671241a11369434175d9c495b6ab4febec093c47d57340c1913c68dc
Jul 23 21:55:34.565688 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.565627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8" event={"ID":"bbe5cad6-2bb2-488f-a91f-14a76d754789","Type":"ContainerStarted","Data":"04369016b28eb05e039287979d1bc84a05e3139e848d631a73cdae0dcfeed8f4"}
Jul 23 21:55:34.565842 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.565744    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8" event={"ID":"bbe5cad6-2bb2-488f-a91f-14a76d754789","Type":"ContainerStarted","Data":"0026877c671241a11369434175d9c495b6ab4febec093c47d57340c1913c68dc"}
Jul 23 21:55:34.791749 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:34.791717    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:35.571376 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:35.571292    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerID="04369016b28eb05e039287979d1bc84a05e3139e848d631a73cdae0dcfeed8f4" exitCode=0
Jul 23 21:55:35.571788 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:35.571377    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8" event={"ID":"bbe5cad6-2bb2-488f-a91f-14a76d754789","Type":"ContainerDied","Data":"04369016b28eb05e039287979d1bc84a05e3139e848d631a73cdae0dcfeed8f4"}
Jul 23 21:55:36.577597 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:36.577511    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerID="caff62cd5d08a0b67bc80ff6f569093aef99441a1fce844f272c2557e476dcb1" exitCode=0
Jul 23 21:55:36.577597 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:36.577575    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8" event={"ID":"bbe5cad6-2bb2-488f-a91f-14a76d754789","Type":"ContainerDied","Data":"caff62cd5d08a0b67bc80ff6f569093aef99441a1fce844f272c2557e476dcb1"}
Jul 23 21:55:37.584417 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:37.584384    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerID="329c19986cf863d8341afe1fdc960c1385fb091a2d1fe25fdc760a9bc7480924" exitCode=0
Jul 23 21:55:37.584817 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:37.584461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8" event={"ID":"bbe5cad6-2bb2-488f-a91f-14a76d754789","Type":"ContainerDied","Data":"329c19986cf863d8341afe1fdc960c1385fb091a2d1fe25fdc760a9bc7480924"}
Jul 23 21:55:38.743457 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.743423    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:38.870448 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.870358    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-util\") pod \"bbe5cad6-2bb2-488f-a91f-14a76d754789\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") "
Jul 23 21:55:38.870448 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.870413    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48r2v\" (UniqueName: \"kubernetes.io/projected/bbe5cad6-2bb2-488f-a91f-14a76d754789-kube-api-access-48r2v\") pod \"bbe5cad6-2bb2-488f-a91f-14a76d754789\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") "
Jul 23 21:55:38.870448 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.870450    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-bundle\") pod \"bbe5cad6-2bb2-488f-a91f-14a76d754789\" (UID: \"bbe5cad6-2bb2-488f-a91f-14a76d754789\") "
Jul 23 21:55:38.871218 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.871192    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-bundle" (OuterVolumeSpecName: "bundle") pod "bbe5cad6-2bb2-488f-a91f-14a76d754789" (UID: "bbe5cad6-2bb2-488f-a91f-14a76d754789"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:55:38.872595 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.872571    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe5cad6-2bb2-488f-a91f-14a76d754789-kube-api-access-48r2v" (OuterVolumeSpecName: "kube-api-access-48r2v") pod "bbe5cad6-2bb2-488f-a91f-14a76d754789" (UID: "bbe5cad6-2bb2-488f-a91f-14a76d754789"). InnerVolumeSpecName "kube-api-access-48r2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:55:38.876226 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.876186    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-util" (OuterVolumeSpecName: "util") pod "bbe5cad6-2bb2-488f-a91f-14a76d754789" (UID: "bbe5cad6-2bb2-488f-a91f-14a76d754789"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:55:38.971418 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.971366    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:38.971418 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.971408    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48r2v\" (UniqueName: \"kubernetes.io/projected/bbe5cad6-2bb2-488f-a91f-14a76d754789-kube-api-access-48r2v\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:38.971418 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:38.971420    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bbe5cad6-2bb2-488f-a91f-14a76d754789-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:55:39.602119 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:39.602082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8" event={"ID":"bbe5cad6-2bb2-488f-a91f-14a76d754789","Type":"ContainerDied","Data":"0026877c671241a11369434175d9c495b6ab4febec093c47d57340c1913c68dc"}
Jul 23 21:55:39.602278 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:39.602154    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0026877c671241a11369434175d9c495b6ab4febec093c47d57340c1913c68dc"
Jul 23 21:55:39.602278 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:39.602103    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbbtq8"
Jul 23 21:55:50.512103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:50.512022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:50.658273 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:50.658236    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:51.154439 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.154396    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:51.428033 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.427955    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:51.530898 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.530856    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:51.563675 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563646    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"]
Jul 23 21:55:51.563950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563936    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="pull"
Jul 23 21:55:51.563992 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563952    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="pull"
Jul 23 21:55:51.563992 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563962    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="util"
Jul 23 21:55:51.563992 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563967    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="util"
Jul 23 21:55:51.563992 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563981    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="extract"
Jul 23 21:55:51.563992 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.563987    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="extract"
Jul 23 21:55:51.564141 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.564039    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe5cad6-2bb2-488f-a91f-14a76d754789" containerName="extract"
Jul 23 21:55:51.568150 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.568134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.571903 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.571877    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 21:55:51.572330 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.572312    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 21:55:51.572524 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.572481    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 21:55:51.572592 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.572535    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 21:55:51.572660 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.572626    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zt4v2\""
Jul 23 21:55:51.591593 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.591569    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"]
Jul 23 21:55:51.667103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.667066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b37abf6a-9a62-44ff-b814-f9ed3096f617-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.667103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.667103    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf6hx\" (UniqueName: \"kubernetes.io/projected/b37abf6a-9a62-44ff-b814-f9ed3096f617-kube-api-access-wf6hx\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.667314 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.667122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b37abf6a-9a62-44ff-b814-f9ed3096f617-tmp\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.667314 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.667195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b37abf6a-9a62-44ff-b814-f9ed3096f617-manifests\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.667314 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.667268    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b37abf6a-9a62-44ff-b814-f9ed3096f617-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.768233 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.768135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b37abf6a-9a62-44ff-b814-f9ed3096f617-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.768233 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.768222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b37abf6a-9a62-44ff-b814-f9ed3096f617-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.768477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.768254    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf6hx\" (UniqueName: \"kubernetes.io/projected/b37abf6a-9a62-44ff-b814-f9ed3096f617-kube-api-access-wf6hx\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.768477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.768278    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b37abf6a-9a62-44ff-b814-f9ed3096f617-tmp\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.768477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.768327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b37abf6a-9a62-44ff-b814-f9ed3096f617-manifests\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.768813 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.768789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b37abf6a-9a62-44ff-b814-f9ed3096f617-manifests\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.770557 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.770526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b37abf6a-9a62-44ff-b814-f9ed3096f617-tmp\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.770705 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.770623    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b37abf6a-9a62-44ff-b814-f9ed3096f617-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.770775 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.770752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b37abf6a-9a62-44ff-b814-f9ed3096f617-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.777290 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.777266    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf6hx\" (UniqueName: \"kubernetes.io/projected/b37abf6a-9a62-44ff-b814-f9ed3096f617-kube-api-access-wf6hx\") pod \"opendatahub-operator-controller-manager-7b7bc97c8c-7blc5\" (UID: \"b37abf6a-9a62-44ff-b814-f9ed3096f617\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.810721 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.810688    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:51.878834 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.878800    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:51.918099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:51.918062    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:52.009114 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:52.009084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:52.024585 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:52.024561    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"]
Jul 23 21:55:52.027476 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:55:52.027447    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37abf6a_9a62_44ff_b814_f9ed3096f617.slice/crio-8790a905ddc01144b1f18abe230cd7c988c5a619200bd036218a9a6f9b9c1c02 WatchSource:0}: Error finding container 8790a905ddc01144b1f18abe230cd7c988c5a619200bd036218a9a6f9b9c1c02: Status 404 returned error can't find the container with id 8790a905ddc01144b1f18abe230cd7c988c5a619200bd036218a9a6f9b9c1c02
Jul 23 21:55:52.111563 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:52.111526    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:52.219491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:52.219459    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:55:52.663548 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:52.663414    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5" event={"ID":"b37abf6a-9a62-44ff-b814-f9ed3096f617","Type":"ContainerStarted","Data":"8790a905ddc01144b1f18abe230cd7c988c5a619200bd036218a9a6f9b9c1c02"}
Jul 23 21:55:55.682726 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:55.682686    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5" event={"ID":"b37abf6a-9a62-44ff-b814-f9ed3096f617","Type":"ContainerStarted","Data":"cd51df552d2acc9988bfe159ceb3d214303c6d8f05d8dfa766f9177dbcc01b32"}
Jul 23 21:55:56.688619 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:56.688581    2575 generic.go:356] "Generic (PLEG): container finished" podID="b37abf6a-9a62-44ff-b814-f9ed3096f617" containerID="cd51df552d2acc9988bfe159ceb3d214303c6d8f05d8dfa766f9177dbcc01b32" exitCode=0
Jul 23 21:55:56.689016 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:56.688671    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5" event={"ID":"b37abf6a-9a62-44ff-b814-f9ed3096f617","Type":"ContainerDied","Data":"cd51df552d2acc9988bfe159ceb3d214303c6d8f05d8dfa766f9177dbcc01b32"}
Jul 23 21:55:57.694679 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:57.694610    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5" event={"ID":"b37abf6a-9a62-44ff-b814-f9ed3096f617","Type":"ContainerStarted","Data":"a2cbb56bb41f2b1572448c3444a6d0a5c2832cffaf156588ffe545227a2f3abc"}
Jul 23 21:55:57.695080 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:57.694731    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:55:57.718366 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:55:57.718319    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5" podStartSLOduration=2.13825738 podStartE2EDuration="6.718303947s" totalImagesPullingTime="3.49694502s" totalInitContainerRuntime="1.083101547s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:55:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:55:57.716744474 +0000 UTC m=+279.977744203" watchObservedRunningTime="2026-07-23 21:55:57.718303947 +0000 UTC m=+279.979303674"
Jul 23 21:56:01.851242 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:01.851201    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:01.951185 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:01.951148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:03.322809 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:03.322764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:03.421016 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:03.420977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:08.700726 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:08.700695    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7bc97c8c-7blc5"
Jul 23 21:56:11.689738 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:11.689684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:11.835430 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:11.835383    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:12.390420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.390379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:12.432191 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.432154    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"]
Jul 23 21:56:12.435210 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.435192    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.439723 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.439697    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 21:56:12.439854 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.439700    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 21:56:12.439854 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.439702    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-glhzz\""
Jul 23 21:56:12.451397 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.451370    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"]
Jul 23 21:56:12.514758 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.514710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:12.551322 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.551288    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-tmp\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.551493 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.551345    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-tls-certs\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.551493 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.551372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgs4d\" (UniqueName: \"kubernetes.io/projected/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-kube-api-access-kgs4d\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.628086 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.628042    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:12.652153 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.652069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-tmp\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.652153 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.652127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-tls-certs\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.652321 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.652156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgs4d\" (UniqueName: \"kubernetes.io/projected/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-kube-api-access-kgs4d\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.654437 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.654404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-tmp\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.654565 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.654546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-tls-certs\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.667706 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.667683    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgs4d\" (UniqueName: \"kubernetes.io/projected/aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998-kube-api-access-kgs4d\") pod \"kube-auth-proxy-5dc6f65d97-79nl4\" (UID: \"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998\") " pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.744829 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.744797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"
Jul 23 21:56:12.758542 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.758509    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:12.861274 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.861235    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:12.886531 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:12.886506    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4"]
Jul 23 21:56:12.888561 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:56:12.888531    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaef0f2b_3df5_4d88_bedc_ca4a6f6ae998.slice/crio-e5f5a5863d2fa62a56a1433d4aba32429c4b03d5a3cdf36936d4912d4a4d7faf WatchSource:0}: Error finding container e5f5a5863d2fa62a56a1433d4aba32429c4b03d5a3cdf36936d4912d4a4d7faf: Status 404 returned error can't find the container with id e5f5a5863d2fa62a56a1433d4aba32429c4b03d5a3cdf36936d4912d4a4d7faf
Jul 23 21:56:13.060958 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.060870    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:13.598334 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.598283    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:13.710495 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.710454    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jkw4d"]
Jul 23 21:56:13.713821 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.713792    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:13.716420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.716358    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 21:56:13.716561 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.716467    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-z56xk\""
Jul 23 21:56:13.731142 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.731004    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jkw4d"]
Jul 23 21:56:13.770355 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.770316    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4" event={"ID":"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998","Type":"ContainerStarted","Data":"e5f5a5863d2fa62a56a1433d4aba32429c4b03d5a3cdf36936d4912d4a4d7faf"}
Jul 23 21:56:13.865855 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.865770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28tg8\" (UniqueName: \"kubernetes.io/projected/19dfbc68-20e8-41d0-809f-a35b8939ee66-kube-api-access-28tg8\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:13.866025 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.865908    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:13.943225 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.943182    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"]
Jul 23 21:56:13.946495 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.946474    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:13.952067 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.952036    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 21:56:13.952236 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.952215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 21:56:13.952298 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.952218    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 21:56:13.952298 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.952284    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 21:56:13.952607 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.952591    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-686hb\""
Jul 23 21:56:13.966869 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.966842    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"]
Jul 23 21:56:13.967129 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.967114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:13.967180 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.967167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28tg8\" (UniqueName: \"kubernetes.io/projected/19dfbc68-20e8-41d0-809f-a35b8939ee66-kube-api-access-28tg8\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:13.967275 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:56:13.967258    2575 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 23 21:56:13.967327 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:56:13.967318    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs podName:19dfbc68-20e8-41d0-809f-a35b8939ee66 nodeName:}" failed. No retries permitted until 2026-07-23 21:56:14.467299156 +0000 UTC m=+296.728298877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs") pod "model-serving-api-54d89bd5fc-jkw4d" (UID: "19dfbc68-20e8-41d0-809f-a35b8939ee66") : secret "model-serving-api-tls" not found
Jul 23 21:56:13.980662 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.980617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:13.983535 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:13.983515    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28tg8\" (UniqueName: \"kubernetes.io/projected/19dfbc68-20e8-41d0-809f-a35b8939ee66-kube-api-access-28tg8\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:14.068161 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-token\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.068336 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.068336 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-cacerts\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.068452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.068452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068389    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.068452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-local-certs\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.068605 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.068451    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5g8p\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-kube-api-access-q5g8p\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.114335 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.114298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:14.169613 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.169522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.169613 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.169569    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.169613 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.169607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-local-certs\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.169913 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.169658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g8p\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-kube-api-access-q5g8p\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.169913 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.169870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-token\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.170027 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.169983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.170084 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.170025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-cacerts\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.170241 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.170219    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.172530 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.172477    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.172658 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.172600    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.172782 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.172653    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-cacerts\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.172903 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.172881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-local-certs\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.180004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.179981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5g8p\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-kube-api-access-q5g8p\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.187119 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.187094    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-token\") pod \"istiod-openshift-gateway-7574459656-rlc25\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.255593 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.255557    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:14.422739 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.422657    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"]
Jul 23 21:56:14.425588 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:56:14.425557    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e05a54_4937_4fc0_b7e5_ac1686d7b4bf.slice/crio-43a7c3c28d47ee07b7b54cf9536a1883c4f1de00d11ee8ade7fcd00bebe8f4ba WatchSource:0}: Error finding container 43a7c3c28d47ee07b7b54cf9536a1883c4f1de00d11ee8ade7fcd00bebe8f4ba: Status 404 returned error can't find the container with id 43a7c3c28d47ee07b7b54cf9536a1883c4f1de00d11ee8ade7fcd00bebe8f4ba
Jul 23 21:56:14.472924 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.472890    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:14.473062 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:56:14.473009    2575 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 23 21:56:14.473062 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:56:14.473058    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs podName:19dfbc68-20e8-41d0-809f-a35b8939ee66 nodeName:}" failed. No retries permitted until 2026-07-23 21:56:15.473045031 +0000 UTC m=+297.734044740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs") pod "model-serving-api-54d89bd5fc-jkw4d" (UID: "19dfbc68-20e8-41d0-809f-a35b8939ee66") : secret "model-serving-api-tls" not found
Jul 23 21:56:14.668311 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.668278    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:14.782866 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:14.782773    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" event={"ID":"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf","Type":"ContainerStarted","Data":"43a7c3c28d47ee07b7b54cf9536a1883c4f1de00d11ee8ade7fcd00bebe8f4ba"}
Jul 23 21:56:15.325874 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.325813    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:15.483596 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.483510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:15.486354 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.486322    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19dfbc68-20e8-41d0-809f-a35b8939ee66-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkw4d\" (UID: \"19dfbc68-20e8-41d0-809f-a35b8939ee66\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:15.526100 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.526064    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:15.790596 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.790557    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4" event={"ID":"aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998","Type":"ContainerStarted","Data":"a7820e2b3cd3a5a6e30d06bea35d53631cd916e618e0f5b27eaba4cf02f1fcb9"}
Jul 23 21:56:15.804109 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.804059    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jkw4d"]
Jul 23 21:56:15.808661 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:15.808513    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dc6f65d97-79nl4" podStartSLOduration=1.49093229 podStartE2EDuration="3.808493882s" totalImagesPullingTime="2.317561592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:56:15.807983374 +0000 UTC m=+298.068983105" watchObservedRunningTime="2026-07-23 21:56:15.808493882 +0000 UTC m=+298.069493609"
Jul 23 21:56:15.809714 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:56:15.809687    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19dfbc68_20e8_41d0_809f_a35b8939ee66.slice/crio-b3939696844d131614a35cce3736352bfb276b5f1843e43184cbd5033f14fe32 WatchSource:0}: Error finding container b3939696844d131614a35cce3736352bfb276b5f1843e43184cbd5033f14fe32: Status 404 returned error can't find the container with id b3939696844d131614a35cce3736352bfb276b5f1843e43184cbd5033f14fe32
Jul 23 21:56:16.798367 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:16.798328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d" event={"ID":"19dfbc68-20e8-41d0-809f-a35b8939ee66","Type":"ContainerStarted","Data":"b3939696844d131614a35cce3736352bfb276b5f1843e43184cbd5033f14fe32"}
Jul 23 21:56:17.105446 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:17.105409    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 21:56:17.105528 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:17.105478    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 23 21:56:17.806217 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:17.806167    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" event={"ID":"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf","Type":"ContainerStarted","Data":"c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0"}
Jul 23 21:56:17.806671 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:17.806238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:17.829567 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:17.829509    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" podStartSLOduration=2.151820317 podStartE2EDuration="4.829488475s" totalImagesPullingTime="2.677668158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:56:17.827072074 +0000 UTC m=+300.088071802" watchObservedRunningTime="2026-07-23 21:56:17.829488475 +0000 UTC m=+300.090488207"
Jul 23 21:56:18.166011 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.165977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:18.187824 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.187776    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 21:56:18.191823 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.191795    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 21:56:18.200116 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.200089    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 21:56:18.261530 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.261367    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:18.757477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.757434    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:18.813121 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.813075    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d" event={"ID":"19dfbc68-20e8-41d0-809f-a35b8939ee66","Type":"ContainerStarted","Data":"71ec53c12aae210232dd850696e33bbeeb36dee0f1d0f44d701d4bd94667040e"}
Jul 23 21:56:18.813547 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.813217    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:18.814940 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.814915    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-rlc25 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 21:56:18.815079 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.814974    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" podUID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 21:56:18.832626 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:18.832574    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d" podStartSLOduration=3.58423759 podStartE2EDuration="5.832561383s" totalImagesPullingTime="2.248323793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:56:18.832128045 +0000 UTC m=+301.093127774" watchObservedRunningTime="2026-07-23 21:56:18.832561383 +0000 UTC m=+301.093561110"
Jul 23 21:56:19.009298 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:19.009199    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:19.109926 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:19.109885    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:21.814041 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:21.814009    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:23.685070 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.685030    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:23.804921 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.804882    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"]
Jul 23 21:56:23.808536 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.808510    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:23.809139 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.809116    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"]
Jul 23 21:56:23.812478 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.812459    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:23.814663 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.814623    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 23 21:56:23.814826 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.814806    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tqcwd\""
Jul 23 21:56:23.818969 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.818952    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 21:56:23.819265 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.819244    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fskvp\""
Jul 23 21:56:23.844299 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.844262    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"]
Jul 23 21:56:23.867326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.867294    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:23.904243 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.904214    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"]
Jul 23 21:56:23.947968 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.947875    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:23.958935 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.958909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdkxq\" (UniqueName: \"kubernetes.io/projected/e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a-kube-api-access-mdkxq\") pod \"kserve-controller-manager-d6f4b65b8-gwc67\" (UID: \"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:23.959057 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.958944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnv2w\" (UniqueName: \"kubernetes.io/projected/1f66ef48-03cc-434d-bd74-8f753afe23a9-kube-api-access-hnv2w\") pod \"llmisvc-controller-manager-777c6bd7d5-xrm96\" (UID: \"1f66ef48-03cc-434d-bd74-8f753afe23a9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:23.959057 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.958996    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f66ef48-03cc-434d-bd74-8f753afe23a9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xrm96\" (UID: \"1f66ef48-03cc-434d-bd74-8f753afe23a9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:23.959057 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:23.959022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a-cert\") pod \"kserve-controller-manager-d6f4b65b8-gwc67\" (UID: \"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:24.054744 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.054710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:24.060001 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.059969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a-cert\") pod \"kserve-controller-manager-d6f4b65b8-gwc67\" (UID: \"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:24.060559 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.060523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdkxq\" (UniqueName: \"kubernetes.io/projected/e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a-kube-api-access-mdkxq\") pod \"kserve-controller-manager-d6f4b65b8-gwc67\" (UID: \"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:24.060724 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.060576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnv2w\" (UniqueName: \"kubernetes.io/projected/1f66ef48-03cc-434d-bd74-8f753afe23a9-kube-api-access-hnv2w\") pod \"llmisvc-controller-manager-777c6bd7d5-xrm96\" (UID: \"1f66ef48-03cc-434d-bd74-8f753afe23a9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:24.060724 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.060698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f66ef48-03cc-434d-bd74-8f753afe23a9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xrm96\" (UID: \"1f66ef48-03cc-434d-bd74-8f753afe23a9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:24.062912 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.062890    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f66ef48-03cc-434d-bd74-8f753afe23a9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xrm96\" (UID: \"1f66ef48-03cc-434d-bd74-8f753afe23a9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:24.063149 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.063131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a-cert\") pod \"kserve-controller-manager-d6f4b65b8-gwc67\" (UID: \"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:24.070879 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.070855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnv2w\" (UniqueName: \"kubernetes.io/projected/1f66ef48-03cc-434d-bd74-8f753afe23a9-kube-api-access-hnv2w\") pod \"llmisvc-controller-manager-777c6bd7d5-xrm96\" (UID: \"1f66ef48-03cc-434d-bd74-8f753afe23a9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:24.071284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.071266    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdkxq\" (UniqueName: \"kubernetes.io/projected/e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a-kube-api-access-mdkxq\") pod \"kserve-controller-manager-d6f4b65b8-gwc67\" (UID: \"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:24.120354 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.120324    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:24.125524 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.125503    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:24.274127 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.274084    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"]
Jul 23 21:56:24.276700 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:56:24.276673    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b9b7a0_ffd3_4b4d_bf26_ad29ee220f9a.slice/crio-c03f59f0b8f63fcbcdfc782227bc1dca708a7316110968be3c94e5fb267665e6 WatchSource:0}: Error finding container c03f59f0b8f63fcbcdfc782227bc1dca708a7316110968be3c94e5fb267665e6: Status 404 returned error can't find the container with id c03f59f0b8f63fcbcdfc782227bc1dca708a7316110968be3c94e5fb267665e6
Jul 23 21:56:24.278030 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.278008    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 21:56:24.297231 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.297209    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"]
Jul 23 21:56:24.298814 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:56:24.298788    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1f66ef48_03cc_434d_bd74_8f753afe23a9.slice/crio-61bfb78880729c71e4c1d6774313a60df07fee10cdd5725130f128be12cb7430 WatchSource:0}: Error finding container 61bfb78880729c71e4c1d6774313a60df07fee10cdd5725130f128be12cb7430: Status 404 returned error can't find the container with id 61bfb78880729c71e4c1d6774313a60df07fee10cdd5725130f128be12cb7430
Jul 23 21:56:24.841084 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.841050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67" event={"ID":"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a","Type":"ContainerStarted","Data":"c03f59f0b8f63fcbcdfc782227bc1dca708a7316110968be3c94e5fb267665e6"}
Jul 23 21:56:24.842033 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:24.842013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96" event={"ID":"1f66ef48-03cc-434d-bd74-8f753afe23a9","Type":"ContainerStarted","Data":"61bfb78880729c71e4c1d6774313a60df07fee10cdd5725130f128be12cb7430"}
Jul 23 21:56:25.059065 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:25.058977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:25.158867 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:25.158803    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:25.259569 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:25.259525    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:27.861054 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:27.861017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96" event={"ID":"1f66ef48-03cc-434d-bd74-8f753afe23a9","Type":"ContainerStarted","Data":"40fff8529c5061cd125ef3ac3e23bb48e2d68ac70846244f937a737696976388"}
Jul 23 21:56:27.861481 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:27.861096    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:27.862503 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:27.862477    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67" event={"ID":"e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a","Type":"ContainerStarted","Data":"0defb05562a226d59ecd3a4ea8b614525a92c66d2e3fe9d03c7f36ad855bfe45"}
Jul 23 21:56:27.862659 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:27.862529    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:56:27.880018 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:27.879972    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96" podStartSLOduration=1.8137165610000001 podStartE2EDuration="4.879957056s" totalImagesPullingTime="3.066240495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:56:27.879706416 +0000 UTC m=+310.140706181" watchObservedRunningTime="2026-07-23 21:56:27.879957056 +0000 UTC m=+310.140956784"
Jul 23 21:56:27.896608 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:27.896564    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67" podStartSLOduration=1.8120504689999999 podStartE2EDuration="4.89654879s" totalImagesPullingTime="3.084498321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:56:27.895878664 +0000 UTC m=+310.156878391" watchObservedRunningTime="2026-07-23 21:56:27.89654879 +0000 UTC m=+310.157548519"
Jul 23 21:56:28.169459 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.169377    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"]
Jul 23 21:56:28.169596 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.169567    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" podUID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" containerName="discovery" containerID="cri-o://c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0" gracePeriod=30
Jul 23 21:56:28.456785 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.456748    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:28.605904 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.605865    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-kubeconfig\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.606099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.605927    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-dns-cert\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.606099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.605987    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5g8p\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-kube-api-access-q5g8p\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.606099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.606022    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-local-certs\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.606099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.606060    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-cacerts\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.606099 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.606088    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-ca-configmap\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.606336 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.606133    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-token\") pod \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\" (UID: \"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf\") "
Jul 23 21:56:28.608097 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.608051    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:56:28.608210 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.608140    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:56:28.609625 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.609593    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:56:28.610041 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.610018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-kube-api-access-q5g8p" (OuterVolumeSpecName: "kube-api-access-q5g8p") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "kube-api-access-q5g8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:56:28.610133 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.610086    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-local-certs" (OuterVolumeSpecName: "local-certs") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:56:28.610173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.610136    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:56:28.610173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.610136    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-cacerts" (OuterVolumeSpecName: "cacerts") pod "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" (UID: "c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706912    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-token\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706940    2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-kubeconfig\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706949    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-dns-cert\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706959    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5g8p\" (UniqueName: \"kubernetes.io/projected/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-kube-api-access-q5g8p\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706968    2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-local-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706976    2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-cacerts\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.706990 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.706983    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf-istio-csr-ca-configmap\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:56:28.868196 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.868162    2575 generic.go:356] "Generic (PLEG): container finished" podID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" containerID="c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0" exitCode=0
Jul 23 21:56:28.868699 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.868221    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"
Jul 23 21:56:28.868699 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.868254    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" event={"ID":"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf","Type":"ContainerDied","Data":"c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0"}
Jul 23 21:56:28.868699 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.868352    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rlc25" event={"ID":"c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf","Type":"ContainerDied","Data":"43a7c3c28d47ee07b7b54cf9536a1883c4f1de00d11ee8ade7fcd00bebe8f4ba"}
Jul 23 21:56:28.868699 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.868385    2575 scope.go:117] "RemoveContainer" containerID="c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0"
Jul 23 21:56:28.878524 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.878505    2575 scope.go:117] "RemoveContainer" containerID="c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0"
Jul 23 21:56:28.878827 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:56:28.878806    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0\": container with ID starting with c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0 not found: ID does not exist" containerID="c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0"
Jul 23 21:56:28.878890 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.878836    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0"} err="failed to get container status \"c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0\": rpc error: code = NotFound desc = could not find container \"c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0\": container with ID starting with c494ae65194caed696fdfc2fb25e6c9f7f4de66b33d99f145a3e3d3d2f65cfe0 not found: ID does not exist"
Jul 23 21:56:28.891902 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.891874    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"]
Jul 23 21:56:28.900745 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:28.900721    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rlc25"]
Jul 23 21:56:29.821978 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:29.821949    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jkw4d"
Jul 23 21:56:30.226804 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:30.226773    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" path="/var/lib/kubelet/pods/c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf/volumes"
Jul 23 21:56:37.637404 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:37.637367    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:39.140039 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:39.140007    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:39.584083 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:39.583984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:39.644723 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:39.644690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:56:58.870660 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:58.870611    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xrm96"
Jul 23 21:56:58.873884 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:56:58.873857    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gwc67"
Jul 23 21:57:13.253028 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.252948    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:13.350408 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.350368    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:13.450519 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.450481    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9bp62"]
Jul 23 21:57:13.450879 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.450865    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" containerName="discovery"
Jul 23 21:57:13.450926 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.450891    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" containerName="discovery"
Jul 23 21:57:13.450963 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.450954    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e05a54-4937-4fc0-b7e5-ac1686d7b4bf" containerName="discovery"
Jul 23 21:57:13.458595 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.458565    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:13.461218 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.461191    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 21:57:13.461366 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.461223    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8s7bv\""
Jul 23 21:57:13.462435 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.462407    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 21:57:13.462977 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.462951    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9bp62"]
Jul 23 21:57:13.502368 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.502325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:13.572712 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.572595    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2lml\" (UniqueName: \"kubernetes.io/projected/c1009cfb-c194-4067-9d20-f0b0a5c97f4f-kube-api-access-r2lml\") pod \"kuadrant-operator-catalog-9bp62\" (UID: \"c1009cfb-c194-4067-9d20-f0b0a5c97f4f\") " pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:13.673549 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.673516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2lml\" (UniqueName: \"kubernetes.io/projected/c1009cfb-c194-4067-9d20-f0b0a5c97f4f-kube-api-access-r2lml\") pod \"kuadrant-operator-catalog-9bp62\" (UID: \"c1009cfb-c194-4067-9d20-f0b0a5c97f4f\") " pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:13.681415 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.681397    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2lml\" (UniqueName: \"kubernetes.io/projected/c1009cfb-c194-4067-9d20-f0b0a5c97f4f-kube-api-access-r2lml\") pod \"kuadrant-operator-catalog-9bp62\" (UID: \"c1009cfb-c194-4067-9d20-f0b0a5c97f4f\") " pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:13.772696 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.772661    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:13.797666 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.797622    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9bp62"]
Jul 23 21:57:13.929181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:13.929145    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9bp62"]
Jul 23 21:57:13.932348 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:13.932306    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1009cfb_c194_4067_9d20_f0b0a5c97f4f.slice/crio-a2eda54ae4a53ae0320cb75400ff169262ad4d2ea4cf421b34a9918b19dc6c93 WatchSource:0}: Error finding container a2eda54ae4a53ae0320cb75400ff169262ad4d2ea4cf421b34a9918b19dc6c93: Status 404 returned error can't find the container with id a2eda54ae4a53ae0320cb75400ff169262ad4d2ea4cf421b34a9918b19dc6c93
Jul 23 21:57:14.008128 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.008095    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-885tg"]
Jul 23 21:57:14.010941 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.010925    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:14.019247 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.019222    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-885tg"]
Jul 23 21:57:14.080220 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.080191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9bp62" event={"ID":"c1009cfb-c194-4067-9d20-f0b0a5c97f4f","Type":"ContainerStarted","Data":"a2eda54ae4a53ae0320cb75400ff169262ad4d2ea4cf421b34a9918b19dc6c93"}
Jul 23 21:57:14.081550 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.081528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrff4\" (UniqueName: \"kubernetes.io/projected/74c91f55-0b19-4063-ae89-630510eed8ff-kube-api-access-qrff4\") pod \"kuadrant-operator-catalog-885tg\" (UID: \"74c91f55-0b19-4063-ae89-630510eed8ff\") " pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:14.182228 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.182138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrff4\" (UniqueName: \"kubernetes.io/projected/74c91f55-0b19-4063-ae89-630510eed8ff-kube-api-access-qrff4\") pod \"kuadrant-operator-catalog-885tg\" (UID: \"74c91f55-0b19-4063-ae89-630510eed8ff\") " pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:14.197570 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.197538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrff4\" (UniqueName: \"kubernetes.io/projected/74c91f55-0b19-4063-ae89-630510eed8ff-kube-api-access-qrff4\") pod \"kuadrant-operator-catalog-885tg\" (UID: \"74c91f55-0b19-4063-ae89-630510eed8ff\") " pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:14.320498 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.320463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:14.470437 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:14.470401    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-885tg"]
Jul 23 21:57:14.490886 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:14.490851    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c91f55_0b19_4063_ae89_630510eed8ff.slice/crio-5df818e65f40025e231bfc19aed0e13838849eb4266b6cba2fd0dd50e60cbc25 WatchSource:0}: Error finding container 5df818e65f40025e231bfc19aed0e13838849eb4266b6cba2fd0dd50e60cbc25: Status 404 returned error can't find the container with id 5df818e65f40025e231bfc19aed0e13838849eb4266b6cba2fd0dd50e60cbc25
Jul 23 21:57:15.087799 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:15.087755    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-885tg" event={"ID":"74c91f55-0b19-4063-ae89-630510eed8ff","Type":"ContainerStarted","Data":"5df818e65f40025e231bfc19aed0e13838849eb4266b6cba2fd0dd50e60cbc25"}
Jul 23 21:57:16.093929 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.093896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-885tg" event={"ID":"74c91f55-0b19-4063-ae89-630510eed8ff","Type":"ContainerStarted","Data":"c1701bf24aeca5cbe61fad4cf1e844c7d710d7813f6d9dadbd77da2863f498a0"}
Jul 23 21:57:16.095372 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.095344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9bp62" event={"ID":"c1009cfb-c194-4067-9d20-f0b0a5c97f4f","Type":"ContainerStarted","Data":"562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23"}
Jul 23 21:57:16.095504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.095370    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9bp62" podUID="c1009cfb-c194-4067-9d20-f0b0a5c97f4f" containerName="registry-server" containerID="cri-o://562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23" gracePeriod=2
Jul 23 21:57:16.118649 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.115553    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-885tg" podStartSLOduration=1.816626694 podStartE2EDuration="3.115533412s" totalImagesPullingTime="1.298906718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:16.112786379 +0000 UTC m=+358.373786109" watchObservedRunningTime="2026-07-23 21:57:16.115533412 +0000 UTC m=+358.376533140"
Jul 23 21:57:16.132968 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.132913    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9bp62" podStartSLOduration=1.278276563 podStartE2EDuration="3.132898675s" totalImagesPullingTime="1.854622112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:16.131949875 +0000 UTC m=+358.392949603" watchObservedRunningTime="2026-07-23 21:57:16.132898675 +0000 UTC m=+358.393898403"
Jul 23 21:57:16.348551 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.348492    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:16.502054 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.502018    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2lml\" (UniqueName: \"kubernetes.io/projected/c1009cfb-c194-4067-9d20-f0b0a5c97f4f-kube-api-access-r2lml\") pod \"c1009cfb-c194-4067-9d20-f0b0a5c97f4f\" (UID: \"c1009cfb-c194-4067-9d20-f0b0a5c97f4f\") "
Jul 23 21:57:16.504167 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.504136    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1009cfb-c194-4067-9d20-f0b0a5c97f4f-kube-api-access-r2lml" (OuterVolumeSpecName: "kube-api-access-r2lml") pod "c1009cfb-c194-4067-9d20-f0b0a5c97f4f" (UID: "c1009cfb-c194-4067-9d20-f0b0a5c97f4f"). InnerVolumeSpecName "kube-api-access-r2lml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:57:16.602804 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:16.602722    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2lml\" (UniqueName: \"kubernetes.io/projected/c1009cfb-c194-4067-9d20-f0b0a5c97f4f-kube-api-access-r2lml\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:17.101798 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.101760    2575 generic.go:356] "Generic (PLEG): container finished" podID="c1009cfb-c194-4067-9d20-f0b0a5c97f4f" containerID="562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23" exitCode=0
Jul 23 21:57:17.102250 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.101802    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9bp62"
Jul 23 21:57:17.102250 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.101842    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9bp62" event={"ID":"c1009cfb-c194-4067-9d20-f0b0a5c97f4f","Type":"ContainerDied","Data":"562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23"}
Jul 23 21:57:17.102250 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.101893    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9bp62" event={"ID":"c1009cfb-c194-4067-9d20-f0b0a5c97f4f","Type":"ContainerDied","Data":"a2eda54ae4a53ae0320cb75400ff169262ad4d2ea4cf421b34a9918b19dc6c93"}
Jul 23 21:57:17.102250 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.101913    2575 scope.go:117] "RemoveContainer" containerID="562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23"
Jul 23 21:57:17.110857 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.110813    2575 scope.go:117] "RemoveContainer" containerID="562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23"
Jul 23 21:57:17.111072 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:57:17.111053    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23\": container with ID starting with 562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23 not found: ID does not exist" containerID="562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23"
Jul 23 21:57:17.111118 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.111081    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23"} err="failed to get container status \"562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23\": rpc error: code = NotFound desc = could not find container \"562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23\": container with ID starting with 562847f6c1cacc3f646c213e52452e5344c3c421988d682dd82d020561a4ab23 not found: ID does not exist"
Jul 23 21:57:17.122980 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.122958    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9bp62"]
Jul 23 21:57:17.127219 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:17.127197    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9bp62"]
Jul 23 21:57:18.226652 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:18.226608    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1009cfb-c194-4067-9d20-f0b0a5c97f4f" path="/var/lib/kubelet/pods/c1009cfb-c194-4067-9d20-f0b0a5c97f4f/volumes"
Jul 23 21:57:24.321279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:24.321238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:24.321888 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:24.321300    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:24.343859 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:24.343824    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:25.161051 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:25.161024    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-885tg"
Jul 23 21:57:28.814222 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.814174    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"]
Jul 23 21:57:28.814654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.814574    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1009cfb-c194-4067-9d20-f0b0a5c97f4f" containerName="registry-server"
Jul 23 21:57:28.814654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.814588    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1009cfb-c194-4067-9d20-f0b0a5c97f4f" containerName="registry-server"
Jul 23 21:57:28.814740 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.814671    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1009cfb-c194-4067-9d20-f0b0a5c97f4f" containerName="registry-server"
Jul 23 21:57:28.817490 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.817470    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:28.819767 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.819744    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4tpmk\""
Jul 23 21:57:28.827923 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:28.827899    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"]
Jul 23 21:57:29.009112 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.009070    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj6tm\" (UniqueName: \"kubernetes.io/projected/a3acd915-4bc2-4815-8d0d-44d81fa2f418-kube-api-access-rj6tm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.009279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.009204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.009279 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.009246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.110083 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.110033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.110232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.110117    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj6tm\" (UniqueName: \"kubernetes.io/projected/a3acd915-4bc2-4815-8d0d-44d81fa2f418-kube-api-access-rj6tm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.110232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.110189    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.110440 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.110422    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.110526 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.110508    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.119451 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.119418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj6tm\" (UniqueName: \"kubernetes.io/projected/a3acd915-4bc2-4815-8d0d-44d81fa2f418-kube-api-access-rj6tm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.126254 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.126229    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:29.261689 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.261664    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"]
Jul 23 21:57:29.263965 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:29.263938    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3acd915_4bc2_4815_8d0d_44d81fa2f418.slice/crio-fe2a0011c904040e58d48ead3ee2d62637fbe15e860586f4fe9a0ac6210b0584 WatchSource:0}: Error finding container fe2a0011c904040e58d48ead3ee2d62637fbe15e860586f4fe9a0ac6210b0584: Status 404 returned error can't find the container with id fe2a0011c904040e58d48ead3ee2d62637fbe15e860586f4fe9a0ac6210b0584
Jul 23 21:57:29.439676 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.439577    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"]
Jul 23 21:57:29.442013 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.441997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.452946 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.452920    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"]
Jul 23 21:57:29.614310 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.614275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fts44\" (UniqueName: \"kubernetes.io/projected/41159bfa-3f68-40c5-b818-34c1c484293d-kube-api-access-fts44\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.614471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.614344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.614471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.614388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.715580 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.715478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fts44\" (UniqueName: \"kubernetes.io/projected/41159bfa-3f68-40c5-b818-34c1c484293d-kube-api-access-fts44\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.715580 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.715555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.715580 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.715574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.715967 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.715947    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.716016 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.715973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.725904 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.725878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fts44\" (UniqueName: \"kubernetes.io/projected/41159bfa-3f68-40c5-b818-34c1c484293d-kube-api-access-fts44\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.751107 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.751066    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:29.887932 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:29.887906    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"]
Jul 23 21:57:29.889941 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:29.889910    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41159bfa_3f68_40c5_b818_34c1c484293d.slice/crio-6ee2394f9efc922092913f475080dcd81a0e6aafa7465b424fead42e765d537b WatchSource:0}: Error finding container 6ee2394f9efc922092913f475080dcd81a0e6aafa7465b424fead42e765d537b: Status 404 returned error can't find the container with id 6ee2394f9efc922092913f475080dcd81a0e6aafa7465b424fead42e765d537b
Jul 23 21:57:30.164059 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.164026    2575 generic.go:356] "Generic (PLEG): container finished" podID="41159bfa-3f68-40c5-b818-34c1c484293d" containerID="0602b9b90a9010566e24f31c39fabe019d14a77929a6be77e450df5cef8d35b2" exitCode=0
Jul 23 21:57:30.164238 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.164098    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr" event={"ID":"41159bfa-3f68-40c5-b818-34c1c484293d","Type":"ContainerDied","Data":"0602b9b90a9010566e24f31c39fabe019d14a77929a6be77e450df5cef8d35b2"}
Jul 23 21:57:30.164238 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.164142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr" event={"ID":"41159bfa-3f68-40c5-b818-34c1c484293d","Type":"ContainerStarted","Data":"6ee2394f9efc922092913f475080dcd81a0e6aafa7465b424fead42e765d537b"}
Jul 23 21:57:30.165601 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.165576    2575 generic.go:356] "Generic (PLEG): container finished" podID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerID="e6cc5bc646573e39f0736916a4e170bdec6c00f3d5a965e92f8265371c07ee36" exitCode=0
Jul 23 21:57:30.165718 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.165652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7" event={"ID":"a3acd915-4bc2-4815-8d0d-44d81fa2f418","Type":"ContainerDied","Data":"e6cc5bc646573e39f0736916a4e170bdec6c00f3d5a965e92f8265371c07ee36"}
Jul 23 21:57:30.165718 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.165702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7" event={"ID":"a3acd915-4bc2-4815-8d0d-44d81fa2f418","Type":"ContainerStarted","Data":"fe2a0011c904040e58d48ead3ee2d62637fbe15e860586f4fe9a0ac6210b0584"}
Jul 23 21:57:30.220280 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.220241    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"]
Jul 23 21:57:30.223165 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.223142    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.232301 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.232276    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"]
Jul 23 21:57:30.321137 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.321043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.321137 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.321125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.321363 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.321187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk8md\" (UniqueName: \"kubernetes.io/projected/a1426ae8-168e-4f18-b921-23138b673f0f-kube-api-access-fk8md\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.422504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.422466    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.422504 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.422505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.422722 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.422536    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk8md\" (UniqueName: \"kubernetes.io/projected/a1426ae8-168e-4f18-b921-23138b673f0f-kube-api-access-fk8md\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.422885 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.422853    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.422926 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.422874    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.437418 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.437383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk8md\" (UniqueName: \"kubernetes.io/projected/a1426ae8-168e-4f18-b921-23138b673f0f-kube-api-access-fk8md\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.533687 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.533651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:30.665682 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.665648    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"]
Jul 23 21:57:30.666728 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:30.666699    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1426ae8_168e_4f18_b921_23138b673f0f.slice/crio-5a6fda958c1fd89c570dd1573654e72c781bbfd52f4889d0d064250c4761461e WatchSource:0}: Error finding container 5a6fda958c1fd89c570dd1573654e72c781bbfd52f4889d0d064250c4761461e: Status 404 returned error can't find the container with id 5a6fda958c1fd89c570dd1573654e72c781bbfd52f4889d0d064250c4761461e
Jul 23 21:57:30.824553 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.824521    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"]
Jul 23 21:57:30.827298 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.827275    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:30.838204 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.838174    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"]
Jul 23 21:57:30.926112 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.926086    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2vc5\" (UniqueName: \"kubernetes.io/projected/40abff84-09dc-4298-93d2-71f43689ac98-kube-api-access-f2vc5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:30.926453 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.926157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:30.926453 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:30.926218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.027113 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.027086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.027232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.027148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.027292 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.027239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2vc5\" (UniqueName: \"kubernetes.io/projected/40abff84-09dc-4298-93d2-71f43689ac98-kube-api-access-f2vc5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.027565 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.027485    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.027717 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.027592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.035579 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.035551    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2vc5\" (UniqueName: \"kubernetes.io/projected/40abff84-09dc-4298-93d2-71f43689ac98-kube-api-access-f2vc5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.137357 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.137329    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:31.181493 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.181457    2575 generic.go:356] "Generic (PLEG): container finished" podID="41159bfa-3f68-40c5-b818-34c1c484293d" containerID="3ac8a8d5ef51ef1a4e7fef5c6100026293b887221ada2af790e3c35eaadc7ab2" exitCode=0
Jul 23 21:57:31.181765 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.181540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr" event={"ID":"41159bfa-3f68-40c5-b818-34c1c484293d","Type":"ContainerDied","Data":"3ac8a8d5ef51ef1a4e7fef5c6100026293b887221ada2af790e3c35eaadc7ab2"}
Jul 23 21:57:31.183682 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.183655    2575 generic.go:356] "Generic (PLEG): container finished" podID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerID="37fba9070a536c8773af44bffd73b1db74a7604c60c42948af9bb20b705ee8f4" exitCode=0
Jul 23 21:57:31.183772 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.183748    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7" event={"ID":"a3acd915-4bc2-4815-8d0d-44d81fa2f418","Type":"ContainerDied","Data":"37fba9070a536c8773af44bffd73b1db74a7604c60c42948af9bb20b705ee8f4"}
Jul 23 21:57:31.185657 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.185614    2575 generic.go:356] "Generic (PLEG): container finished" podID="a1426ae8-168e-4f18-b921-23138b673f0f" containerID="022ac7ffecc68d0a547516347da31ba1f2ee13db5300ba5590190342b369a78b" exitCode=0
Jul 23 21:57:31.185783 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.185659    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6" event={"ID":"a1426ae8-168e-4f18-b921-23138b673f0f","Type":"ContainerDied","Data":"022ac7ffecc68d0a547516347da31ba1f2ee13db5300ba5590190342b369a78b"}
Jul 23 21:57:31.185783 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.185700    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6" event={"ID":"a1426ae8-168e-4f18-b921-23138b673f0f","Type":"ContainerStarted","Data":"5a6fda958c1fd89c570dd1573654e72c781bbfd52f4889d0d064250c4761461e"}
Jul 23 21:57:31.289359 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:31.289322    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"]
Jul 23 21:57:31.290402 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:31.290367    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40abff84_09dc_4298_93d2_71f43689ac98.slice/crio-ed59fdd0746272d89461deeae12cbb5504fcbd5a794978938e4d1e5a0c631764 WatchSource:0}: Error finding container ed59fdd0746272d89461deeae12cbb5504fcbd5a794978938e4d1e5a0c631764: Status 404 returned error can't find the container with id ed59fdd0746272d89461deeae12cbb5504fcbd5a794978938e4d1e5a0c631764
Jul 23 21:57:32.193775 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.193724    2575 generic.go:356] "Generic (PLEG): container finished" podID="40abff84-09dc-4298-93d2-71f43689ac98" containerID="0594f067a14b0c67a845805e36c2cd38302644bfa6abfa236478d988a4a1ba55" exitCode=0
Jul 23 21:57:32.194195 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.193813    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp" event={"ID":"40abff84-09dc-4298-93d2-71f43689ac98","Type":"ContainerDied","Data":"0594f067a14b0c67a845805e36c2cd38302644bfa6abfa236478d988a4a1ba55"}
Jul 23 21:57:32.194195 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.193875    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp" event={"ID":"40abff84-09dc-4298-93d2-71f43689ac98","Type":"ContainerStarted","Data":"ed59fdd0746272d89461deeae12cbb5504fcbd5a794978938e4d1e5a0c631764"}
Jul 23 21:57:32.196140 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.196115    2575 generic.go:356] "Generic (PLEG): container finished" podID="41159bfa-3f68-40c5-b818-34c1c484293d" containerID="d02dd2d44c73004a7966f663fcf4a59672b2680a411dd64454c5a31bc88fcdcc" exitCode=0
Jul 23 21:57:32.196228 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.196191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr" event={"ID":"41159bfa-3f68-40c5-b818-34c1c484293d","Type":"ContainerDied","Data":"d02dd2d44c73004a7966f663fcf4a59672b2680a411dd64454c5a31bc88fcdcc"}
Jul 23 21:57:32.198275 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.198250    2575 generic.go:356] "Generic (PLEG): container finished" podID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerID="06cd654297f1cad5339f1e0c8e8af64d38a8938406c854b031d0c914d1dd45c7" exitCode=0
Jul 23 21:57:32.198365 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.198329    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7" event={"ID":"a3acd915-4bc2-4815-8d0d-44d81fa2f418","Type":"ContainerDied","Data":"06cd654297f1cad5339f1e0c8e8af64d38a8938406c854b031d0c914d1dd45c7"}
Jul 23 21:57:32.200078 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.200057    2575 generic.go:356] "Generic (PLEG): container finished" podID="a1426ae8-168e-4f18-b921-23138b673f0f" containerID="298a837d55fca01deff7548e771cc77198d9286607df01428dd0cb0992ea40ac" exitCode=0
Jul 23 21:57:32.200159 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:32.200117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6" event={"ID":"a1426ae8-168e-4f18-b921-23138b673f0f","Type":"ContainerDied","Data":"298a837d55fca01deff7548e771cc77198d9286607df01428dd0cb0992ea40ac"}
Jul 23 21:57:33.207070 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.206977    2575 generic.go:356] "Generic (PLEG): container finished" podID="a1426ae8-168e-4f18-b921-23138b673f0f" containerID="ad8f2b4b3e8e4712e81c1f0d73af9eac6b2d310316a136cae3741e2996586938" exitCode=0
Jul 23 21:57:33.207070 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.207057    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6" event={"ID":"a1426ae8-168e-4f18-b921-23138b673f0f","Type":"ContainerDied","Data":"ad8f2b4b3e8e4712e81c1f0d73af9eac6b2d310316a136cae3741e2996586938"}
Jul 23 21:57:33.208689 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.208665    2575 generic.go:356] "Generic (PLEG): container finished" podID="40abff84-09dc-4298-93d2-71f43689ac98" containerID="f24eb27f950172b4e7230b51f94baf328e5ed4fe5eb58bae9f2b95edb0909d20" exitCode=0
Jul 23 21:57:33.208822 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.208734    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp" event={"ID":"40abff84-09dc-4298-93d2-71f43689ac98","Type":"ContainerDied","Data":"f24eb27f950172b4e7230b51f94baf328e5ed4fe5eb58bae9f2b95edb0909d20"}
Jul 23 21:57:33.358925 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.358897    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:33.420388 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.420357    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:33.448103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448068    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-bundle\") pod \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") "
Jul 23 21:57:33.448103 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448110    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fts44\" (UniqueName: \"kubernetes.io/projected/41159bfa-3f68-40c5-b818-34c1c484293d-kube-api-access-fts44\") pod \"41159bfa-3f68-40c5-b818-34c1c484293d\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") "
Jul 23 21:57:33.448302 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448148    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj6tm\" (UniqueName: \"kubernetes.io/projected/a3acd915-4bc2-4815-8d0d-44d81fa2f418-kube-api-access-rj6tm\") pod \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") "
Jul 23 21:57:33.448302 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448167    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-util\") pod \"41159bfa-3f68-40c5-b818-34c1c484293d\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") "
Jul 23 21:57:33.448302 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448218    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-util\") pod \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\" (UID: \"a3acd915-4bc2-4815-8d0d-44d81fa2f418\") "
Jul 23 21:57:33.448446 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-bundle\") pod \"41159bfa-3f68-40c5-b818-34c1c484293d\" (UID: \"41159bfa-3f68-40c5-b818-34c1c484293d\") "
Jul 23 21:57:33.448872 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.448841    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-bundle" (OuterVolumeSpecName: "bundle") pod "a3acd915-4bc2-4815-8d0d-44d81fa2f418" (UID: "a3acd915-4bc2-4815-8d0d-44d81fa2f418"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:33.449188 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.449145    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-bundle" (OuterVolumeSpecName: "bundle") pod "41159bfa-3f68-40c5-b818-34c1c484293d" (UID: "41159bfa-3f68-40c5-b818-34c1c484293d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:33.450370 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.450342    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41159bfa-3f68-40c5-b818-34c1c484293d-kube-api-access-fts44" (OuterVolumeSpecName: "kube-api-access-fts44") pod "41159bfa-3f68-40c5-b818-34c1c484293d" (UID: "41159bfa-3f68-40c5-b818-34c1c484293d"). InnerVolumeSpecName "kube-api-access-fts44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:57:33.450813 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.450795    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3acd915-4bc2-4815-8d0d-44d81fa2f418-kube-api-access-rj6tm" (OuterVolumeSpecName: "kube-api-access-rj6tm") pod "a3acd915-4bc2-4815-8d0d-44d81fa2f418" (UID: "a3acd915-4bc2-4815-8d0d-44d81fa2f418"). InnerVolumeSpecName "kube-api-access-rj6tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:57:33.454046 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.454014    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-util" (OuterVolumeSpecName: "util") pod "a3acd915-4bc2-4815-8d0d-44d81fa2f418" (UID: "a3acd915-4bc2-4815-8d0d-44d81fa2f418"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:33.454340 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.454318    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-util" (OuterVolumeSpecName: "util") pod "41159bfa-3f68-40c5-b818-34c1c484293d" (UID: "41159bfa-3f68-40c5-b818-34c1c484293d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:33.549897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.549815    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:33.549897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.549843    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:33.549897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.549852    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3acd915-4bc2-4815-8d0d-44d81fa2f418-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:33.549897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.549860    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fts44\" (UniqueName: \"kubernetes.io/projected/41159bfa-3f68-40c5-b818-34c1c484293d-kube-api-access-fts44\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:33.549897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.549870    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj6tm\" (UniqueName: \"kubernetes.io/projected/a3acd915-4bc2-4815-8d0d-44d81fa2f418-kube-api-access-rj6tm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:33.549897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:33.549878    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41159bfa-3f68-40c5-b818-34c1c484293d-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:34.215663 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.215611    2575 generic.go:356] "Generic (PLEG): container finished" podID="40abff84-09dc-4298-93d2-71f43689ac98" containerID="be3b5e50864fefcb6f90ba79f6e9f7f14fa4f54ce9027c7ab25e5ee6ea9881b9" exitCode=0
Jul 23 21:57:34.216085 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.215679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp" event={"ID":"40abff84-09dc-4298-93d2-71f43689ac98","Type":"ContainerDied","Data":"be3b5e50864fefcb6f90ba79f6e9f7f14fa4f54ce9027c7ab25e5ee6ea9881b9"}
Jul 23 21:57:34.217537 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.217514    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr" event={"ID":"41159bfa-3f68-40c5-b818-34c1c484293d","Type":"ContainerDied","Data":"6ee2394f9efc922092913f475080dcd81a0e6aafa7465b424fead42e765d537b"}
Jul 23 21:57:34.217537 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.217527    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr"
Jul 23 21:57:34.217727 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.217549    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ee2394f9efc922092913f475080dcd81a0e6aafa7465b424fead42e765d537b"
Jul 23 21:57:34.219495 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.219479    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7"
Jul 23 21:57:34.219605 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.219516    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7" event={"ID":"a3acd915-4bc2-4815-8d0d-44d81fa2f418","Type":"ContainerDied","Data":"fe2a0011c904040e58d48ead3ee2d62637fbe15e860586f4fe9a0ac6210b0584"}
Jul 23 21:57:34.219605 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.219555    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe2a0011c904040e58d48ead3ee2d62637fbe15e860586f4fe9a0ac6210b0584"
Jul 23 21:57:34.360167 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.360142    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:34.459165 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.459112    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-util\") pod \"a1426ae8-168e-4f18-b921-23138b673f0f\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") "
Jul 23 21:57:34.459165 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.459180    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-bundle\") pod \"a1426ae8-168e-4f18-b921-23138b673f0f\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") "
Jul 23 21:57:34.459422 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.459216    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk8md\" (UniqueName: \"kubernetes.io/projected/a1426ae8-168e-4f18-b921-23138b673f0f-kube-api-access-fk8md\") pod \"a1426ae8-168e-4f18-b921-23138b673f0f\" (UID: \"a1426ae8-168e-4f18-b921-23138b673f0f\") "
Jul 23 21:57:34.459710 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.459686    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-bundle" (OuterVolumeSpecName: "bundle") pod "a1426ae8-168e-4f18-b921-23138b673f0f" (UID: "a1426ae8-168e-4f18-b921-23138b673f0f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:34.461344 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.461320    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1426ae8-168e-4f18-b921-23138b673f0f-kube-api-access-fk8md" (OuterVolumeSpecName: "kube-api-access-fk8md") pod "a1426ae8-168e-4f18-b921-23138b673f0f" (UID: "a1426ae8-168e-4f18-b921-23138b673f0f"). InnerVolumeSpecName "kube-api-access-fk8md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:57:34.465049 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.465012    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-util" (OuterVolumeSpecName: "util") pod "a1426ae8-168e-4f18-b921-23138b673f0f" (UID: "a1426ae8-168e-4f18-b921-23138b673f0f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:34.560333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.560278    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk8md\" (UniqueName: \"kubernetes.io/projected/a1426ae8-168e-4f18-b921-23138b673f0f-kube-api-access-fk8md\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:34.560333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.560325    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:34.560333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:34.560335    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1426ae8-168e-4f18-b921-23138b673f0f-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:35.225862 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.225831    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6"
Jul 23 21:57:35.226329 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.225861    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6" event={"ID":"a1426ae8-168e-4f18-b921-23138b673f0f","Type":"ContainerDied","Data":"5a6fda958c1fd89c570dd1573654e72c781bbfd52f4889d0d064250c4761461e"}
Jul 23 21:57:35.226329 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.225901    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a6fda958c1fd89c570dd1573654e72c781bbfd52f4889d0d064250c4761461e"
Jul 23 21:57:35.355319 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.355292    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:35.366249 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.366212    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2vc5\" (UniqueName: \"kubernetes.io/projected/40abff84-09dc-4298-93d2-71f43689ac98-kube-api-access-f2vc5\") pod \"40abff84-09dc-4298-93d2-71f43689ac98\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") "
Jul 23 21:57:35.366422 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.366268    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-bundle\") pod \"40abff84-09dc-4298-93d2-71f43689ac98\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") "
Jul 23 21:57:35.366422 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.366296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-util\") pod \"40abff84-09dc-4298-93d2-71f43689ac98\" (UID: \"40abff84-09dc-4298-93d2-71f43689ac98\") "
Jul 23 21:57:35.366708 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.366685    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-bundle" (OuterVolumeSpecName: "bundle") pod "40abff84-09dc-4298-93d2-71f43689ac98" (UID: "40abff84-09dc-4298-93d2-71f43689ac98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:35.368274 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.368245    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40abff84-09dc-4298-93d2-71f43689ac98-kube-api-access-f2vc5" (OuterVolumeSpecName: "kube-api-access-f2vc5") pod "40abff84-09dc-4298-93d2-71f43689ac98" (UID: "40abff84-09dc-4298-93d2-71f43689ac98"). InnerVolumeSpecName "kube-api-access-f2vc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:57:35.373797 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.373755    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-util" (OuterVolumeSpecName: "util") pod "40abff84-09dc-4298-93d2-71f43689ac98" (UID: "40abff84-09dc-4298-93d2-71f43689ac98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:57:35.467258 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.467222    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2vc5\" (UniqueName: \"kubernetes.io/projected/40abff84-09dc-4298-93d2-71f43689ac98-kube-api-access-f2vc5\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:35.467258 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.467251    2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:35.467258 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:35.467261    2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40abff84-09dc-4298-93d2-71f43689ac98-util\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:57:36.233450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:36.233419    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp"
Jul 23 21:57:36.233450 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:36.233438    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp" event={"ID":"40abff84-09dc-4298-93d2-71f43689ac98","Type":"ContainerDied","Data":"ed59fdd0746272d89461deeae12cbb5504fcbd5a794978938e4d1e5a0c631764"}
Jul 23 21:57:36.233880 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:36.233483    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed59fdd0746272d89461deeae12cbb5504fcbd5a794978938e4d1e5a0c631764"
Jul 23 21:57:42.952899 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:42.952863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:46.617357 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:46.617318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:47.686466 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686432    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"]
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686768    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="pull"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686779    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="pull"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686792    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="extract"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686798    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="extract"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686806    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="pull"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686812    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="pull"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686819    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="util"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686824    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="util"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686829    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="util"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686834    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="util"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686841    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="extract"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686845    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="extract"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686853    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="pull"
Jul 23 21:57:47.686850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686858    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="pull"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686866    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="util"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686871    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="util"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686879    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="util"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686883    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="util"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686888    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="pull"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686893    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="pull"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686904    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686908    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686913    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686918    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686965    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3acd915-4bc2-4815-8d0d-44d81fa2f418" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686972    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1426ae8-168e-4f18-b921-23138b673f0f" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686979    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="40abff84-09dc-4298-93d2-71f43689ac98" containerName="extract"
Jul 23 21:57:47.687272 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.686985    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="41159bfa-3f68-40c5-b818-34c1c484293d" containerName="extract"
Jul 23 21:57:47.693334 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.693317    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:47.695899 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.695879    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r28lz\""
Jul 23 21:57:47.704303 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.704281    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"]
Jul 23 21:57:47.777438 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.777403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b3d5e35-b400-4e10-8931-81c8d79ecc00-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:47.777577 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.777486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khhql\" (UniqueName: \"kubernetes.io/projected/1b3d5e35-b400-4e10-8931-81c8d79ecc00-kube-api-access-khhql\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:47.878286 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.878246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khhql\" (UniqueName: \"kubernetes.io/projected/1b3d5e35-b400-4e10-8931-81c8d79ecc00-kube-api-access-khhql\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:47.878433 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.878299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b3d5e35-b400-4e10-8931-81c8d79ecc00-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:47.878732 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.878707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b3d5e35-b400-4e10-8931-81c8d79ecc00-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:47.899950 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:47.899924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khhql\" (UniqueName: \"kubernetes.io/projected/1b3d5e35-b400-4e10-8931-81c8d79ecc00-kube-api-access-khhql\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:48.003880 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:48.003791    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:48.144288 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:48.144263    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"]
Jul 23 21:57:48.146762 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:48.146731    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3d5e35_b400_4e10_8931_81c8d79ecc00.slice/crio-241ed4839cd3459cd55108e9a2750dccfe9cb8ee58889c9399f935b3dc35aa6c WatchSource:0}: Error finding container 241ed4839cd3459cd55108e9a2750dccfe9cb8ee58889c9399f935b3dc35aa6c: Status 404 returned error can't find the container with id 241ed4839cd3459cd55108e9a2750dccfe9cb8ee58889c9399f935b3dc35aa6c
Jul 23 21:57:48.295674 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:48.295562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7" event={"ID":"1b3d5e35-b400-4e10-8931-81c8d79ecc00","Type":"ContainerStarted","Data":"241ed4839cd3459cd55108e9a2750dccfe9cb8ee58889c9399f935b3dc35aa6c"}
Jul 23 21:57:48.373283 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:48.373248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:49.650387 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.650345    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"]
Jul 23 21:57:49.654717 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.654411    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:57:49.658881 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.658857    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 21:57:49.659031 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.658929    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ml7qx\""
Jul 23 21:57:49.664444 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.664420    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"]
Jul 23 21:57:49.796460 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.796417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9qh\" (UniqueName: \"kubernetes.io/projected/779ad542-316e-4ba6-a1a9-4581bf9a4c99-kube-api-access-zj9qh\") pod \"dns-operator-controller-manager-864dd78c4c-qz828\" (UID: \"779ad542-316e-4ba6-a1a9-4581bf9a4c99\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:57:49.897759 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.897718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9qh\" (UniqueName: \"kubernetes.io/projected/779ad542-316e-4ba6-a1a9-4581bf9a4c99-kube-api-access-zj9qh\") pod \"dns-operator-controller-manager-864dd78c4c-qz828\" (UID: \"779ad542-316e-4ba6-a1a9-4581bf9a4c99\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:57:49.917087 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.917025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9qh\" (UniqueName: \"kubernetes.io/projected/779ad542-316e-4ba6-a1a9-4581bf9a4c99-kube-api-access-zj9qh\") pod \"dns-operator-controller-manager-864dd78c4c-qz828\" (UID: \"779ad542-316e-4ba6-a1a9-4581bf9a4c99\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:57:49.969599 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:49.969551    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:57:50.121534 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:50.121504    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"]
Jul 23 21:57:50.123796 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:50.123764    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779ad542_316e_4ba6_a1a9_4581bf9a4c99.slice/crio-9469e96c440fa3d8b0a6ca91b30fd75e9502454088380d6e8e2d6c9380b2f7ce WatchSource:0}: Error finding container 9469e96c440fa3d8b0a6ca91b30fd75e9502454088380d6e8e2d6c9380b2f7ce: Status 404 returned error can't find the container with id 9469e96c440fa3d8b0a6ca91b30fd75e9502454088380d6e8e2d6c9380b2f7ce
Jul 23 21:57:50.197982 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:50.197896    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:50.309252 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:50.309210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828" event={"ID":"779ad542-316e-4ba6-a1a9-4581bf9a4c99","Type":"ContainerStarted","Data":"9469e96c440fa3d8b0a6ca91b30fd75e9502454088380d6e8e2d6c9380b2f7ce"}
Jul 23 21:57:50.363973 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:50.363939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:52.570897 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.570863    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"]
Jul 23 21:57:52.574420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.574398    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:57:52.577503 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.577478    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-l5kzk\""
Jul 23 21:57:52.586215 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.586190    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"]
Jul 23 21:57:52.612315 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.612281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:52.726323 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.726272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5zhr\" (UniqueName: \"kubernetes.io/projected/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c-kube-api-access-n5zhr\") pod \"limitador-operator-controller-manager-859d7b78bd-82dw8\" (UID: \"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:57:52.772791 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.772755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:52.827345 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.827251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5zhr\" (UniqueName: \"kubernetes.io/projected/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c-kube-api-access-n5zhr\") pod \"limitador-operator-controller-manager-859d7b78bd-82dw8\" (UID: \"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:57:52.846004 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.845970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5zhr\" (UniqueName: \"kubernetes.io/projected/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c-kube-api-access-n5zhr\") pod \"limitador-operator-controller-manager-859d7b78bd-82dw8\" (UID: \"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:57:52.886535 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:52.886495    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:57:53.065297 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.065257    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"]
Jul 23 21:57:53.067426 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:53.067396    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9dd2ea7_ebe7_400a_97c3_aa89398bd81c.slice/crio-ac9cc19ca858d9d2133b74c8c9af1fcc849290a47cba8ce93b47214a3d9d4c80 WatchSource:0}: Error finding container ac9cc19ca858d9d2133b74c8c9af1fcc849290a47cba8ce93b47214a3d9d4c80: Status 404 returned error can't find the container with id ac9cc19ca858d9d2133b74c8c9af1fcc849290a47cba8ce93b47214a3d9d4c80
Jul 23 21:57:53.330820 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.330781    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828" event={"ID":"779ad542-316e-4ba6-a1a9-4581bf9a4c99","Type":"ContainerStarted","Data":"afe68fe6a20256f38f54339d2d1251b43fe1a6c2cbec196c4e9c3f3ccfb48dec"}
Jul 23 21:57:53.331064 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.330916    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:57:53.333009 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.332977    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7" event={"ID":"1b3d5e35-b400-4e10-8931-81c8d79ecc00","Type":"ContainerStarted","Data":"18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237"}
Jul 23 21:57:53.333138 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.333057    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:57:53.334500 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.334472    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8" event={"ID":"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c","Type":"ContainerStarted","Data":"ac9cc19ca858d9d2133b74c8c9af1fcc849290a47cba8ce93b47214a3d9d4c80"}
Jul 23 21:57:53.357085 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.357026    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828" podStartSLOduration=1.525221385 podStartE2EDuration="4.357010544s" totalImagesPullingTime="2.831789159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:53.353058613 +0000 UTC m=+395.614058341" watchObservedRunningTime="2026-07-23 21:57:53.357010544 +0000 UTC m=+395.618010271"
Jul 23 21:57:53.391978 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.391914    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7" podStartSLOduration=1.620902171 podStartE2EDuration="6.391895676s" totalImagesPullingTime="4.770993505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:53.388821331 +0000 UTC m=+395.649821100" watchObservedRunningTime="2026-07-23 21:57:53.391895676 +0000 UTC m=+395.652895404"
Jul 23 21:57:53.586081 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:53.585961    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:55.023575 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.023539    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:55.348438 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.348396    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8" event={"ID":"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c","Type":"ContainerStarted","Data":"26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430"}
Jul 23 21:57:55.348756 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.348470    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:57:55.369535 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.369485    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8" podStartSLOduration=2.094641019 podStartE2EDuration="3.369469757s" totalImagesPullingTime="1.274828738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:55.368285577 +0000 UTC m=+397.629285325" watchObservedRunningTime="2026-07-23 21:57:55.369469757 +0000 UTC m=+397.630469485"
Jul 23 21:57:55.893544 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.893511    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"]
Jul 23 21:57:55.897045 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.897025    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:55.899207 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.899167    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 21:57:55.899207 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.899191    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4tpmk\""
Jul 23 21:57:55.899347 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.899306    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 21:57:55.909022 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.909000    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"]
Jul 23 21:57:55.916059 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:55.916024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:56.043270 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.043220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:56.044539 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.044515    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"]
Jul 23 21:57:56.047870 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.047850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:57:56.050240 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.050222    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ldbpp\""
Jul 23 21:57:56.054761 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.054731    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.054867 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.054779    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtbnw\" (UniqueName: \"kubernetes.io/projected/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-kube-api-access-dtbnw\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.054867 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.054800    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.061757 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.061723    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"]
Jul 23 21:57:56.156220 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.156124    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.156220 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.156175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtbnw\" (UniqueName: \"kubernetes.io/projected/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-kube-api-access-dtbnw\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.156220 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.156206    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.156518 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.156261    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb7x6\" (UniqueName: \"kubernetes.io/projected/810defa1-272c-48eb-85a4-5e6138be1e34-kube-api-access-wb7x6\") pod \"authorino-operator-75cc8d5cb7-z9k6h\" (UID: \"810defa1-272c-48eb-85a4-5e6138be1e34\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:57:56.157054 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.157026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.158715 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.158692    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.164160 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.164140    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtbnw\" (UniqueName: \"kubernetes.io/projected/bfd8d6b4-d360-43d4-b11f-793a9d63b9a3-kube-api-access-dtbnw\") pod \"kuadrant-console-plugin-66b5774985-2ncs6\" (UID: \"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.205850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.205817    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"
Jul 23 21:57:56.257936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.257712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb7x6\" (UniqueName: \"kubernetes.io/projected/810defa1-272c-48eb-85a4-5e6138be1e34-kube-api-access-wb7x6\") pod \"authorino-operator-75cc8d5cb7-z9k6h\" (UID: \"810defa1-272c-48eb-85a4-5e6138be1e34\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:57:56.277098 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.277068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb7x6\" (UniqueName: \"kubernetes.io/projected/810defa1-272c-48eb-85a4-5e6138be1e34-kube-api-access-wb7x6\") pod \"authorino-operator-75cc8d5cb7-z9k6h\" (UID: \"810defa1-272c-48eb-85a4-5e6138be1e34\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:57:56.357889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.357850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:57:56.365135 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.365110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6"]
Jul 23 21:57:56.369222 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:56.369192    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfd8d6b4_d360_43d4_b11f_793a9d63b9a3.slice/crio-d26e69901ea31edfb7af85b7fe0228629bc8e4d536b3570a0377b4fd33c7dd30 WatchSource:0}: Error finding container d26e69901ea31edfb7af85b7fe0228629bc8e4d536b3570a0377b4fd33c7dd30: Status 404 returned error can't find the container with id d26e69901ea31edfb7af85b7fe0228629bc8e4d536b3570a0377b4fd33c7dd30
Jul 23 21:57:56.499226 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:56.499198    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"]
Jul 23 21:57:56.500973 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:57:56.500947    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod810defa1_272c_48eb_85a4_5e6138be1e34.slice/crio-3602e76308d4d9c1a5702a92d2ed604ddce5855ba9b7dcb8ee3a7fd12707524e WatchSource:0}: Error finding container 3602e76308d4d9c1a5702a92d2ed604ddce5855ba9b7dcb8ee3a7fd12707524e: Status 404 returned error can't find the container with id 3602e76308d4d9c1a5702a92d2ed604ddce5855ba9b7dcb8ee3a7fd12707524e
Jul 23 21:57:57.363732 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:57.363669    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6" event={"ID":"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3","Type":"ContainerStarted","Data":"d26e69901ea31edfb7af85b7fe0228629bc8e4d536b3570a0377b4fd33c7dd30"}
Jul 23 21:57:57.367157 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:57.367083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h" event={"ID":"810defa1-272c-48eb-85a4-5e6138be1e34","Type":"ContainerStarted","Data":"3602e76308d4d9c1a5702a92d2ed604ddce5855ba9b7dcb8ee3a7fd12707524e"}
Jul 23 21:57:57.978974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:57.978888    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:57:58.085518 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:57:58.085476    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:01.396021 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:01.395979    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h" event={"ID":"810defa1-272c-48eb-85a4-5e6138be1e34","Type":"ContainerStarted","Data":"36d95e048ce5e1373ac310e5e43cd9a45fab5dd11a0a39a73732f145e5c7a9a6"}
Jul 23 21:58:01.396460 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:01.396055    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:58:01.420147 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:01.420098    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h" podStartSLOduration=0.891830405 podStartE2EDuration="5.420082068s" totalImagesPullingTime="4.528251663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:01.419392571 +0000 UTC m=+403.680392298" watchObservedRunningTime="2026-07-23 21:58:01.420082068 +0000 UTC m=+403.681082133"
Jul 23 21:58:04.343454 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:04.343424    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qz828"
Jul 23 21:58:04.343914 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:04.343775    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:05.159654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.159607    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6986648f48-f86gf"]
Jul 23 21:58:05.167070 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.167040    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.181697 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.181665    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6986648f48-f86gf"]
Jul 23 21:58:05.262080 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.262043    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:05.345166 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345131    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0da43441-b5b6-4837-9adf-4a6407d7c859-console-oauth-config\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.345606 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-console-config\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.345606 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s78pk\" (UniqueName: \"kubernetes.io/projected/0da43441-b5b6-4837-9adf-4a6407d7c859-kube-api-access-s78pk\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.345606 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-trusted-ca-bundle\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.345606 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345288    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-oauth-serving-cert\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.345606 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0da43441-b5b6-4837-9adf-4a6407d7c859-console-serving-cert\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.345606 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.345351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-service-ca\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.368799 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.368758    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:05.446649 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446547    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0da43441-b5b6-4837-9adf-4a6407d7c859-console-oauth-config\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.446649 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-console-config\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.446649 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446606    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s78pk\" (UniqueName: \"kubernetes.io/projected/0da43441-b5b6-4837-9adf-4a6407d7c859-kube-api-access-s78pk\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.446936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-trusted-ca-bundle\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.446936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446702    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-oauth-serving-cert\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.446936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446760    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0da43441-b5b6-4837-9adf-4a6407d7c859-console-serving-cert\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.446936 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.446798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-service-ca\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.447463 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.447413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-console-config\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.447809 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.447784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-service-ca\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.447876 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.447859    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-oauth-serving-cert\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.448014 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.447984    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0da43441-b5b6-4837-9adf-4a6407d7c859-trusted-ca-bundle\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.449557 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.449536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0da43441-b5b6-4837-9adf-4a6407d7c859-console-serving-cert\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.449698 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.449655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0da43441-b5b6-4837-9adf-4a6407d7c859-console-oauth-config\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.465732 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.465696    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s78pk\" (UniqueName: \"kubernetes.io/projected/0da43441-b5b6-4837-9adf-4a6407d7c859-kube-api-access-s78pk\") pod \"console-6986648f48-f86gf\" (UID: \"0da43441-b5b6-4837-9adf-4a6407d7c859\") " pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.479795 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.479751    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:05.742431 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.742333    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:05.906900 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:05.906827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:06.228989 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.228954    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"]
Jul 23 21:58:06.229191 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.229156    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" containerName="manager" containerID="cri-o://18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237" gracePeriod=2
Jul 23 21:58:06.234919 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.234873    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:06.236213 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.236182    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"]
Jul 23 21:58:06.252580 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.252540    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"]
Jul 23 21:58:06.252788 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.252762    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" containerName="manager" containerID="cri-o://26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430" gracePeriod=2
Jul 23 21:58:06.254998 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.254963    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:06.255206 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.255189    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:06.267709 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.267678    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"]
Jul 23 21:58:06.268162 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.268141    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" containerName="manager"
Jul 23 21:58:06.268273 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.268166    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" containerName="manager"
Jul 23 21:58:06.268333 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.268273    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" containerName="manager"
Jul 23 21:58:06.273734 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.273322    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.274699 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.274677    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"]
Jul 23 21:58:06.283484 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.283447    2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-82dw8\": pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:06.294953 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.293941    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"]
Jul 23 21:58:06.294953 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.294468    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" containerName="manager"
Jul 23 21:58:06.294953 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.294489    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" containerName="manager"
Jul 23 21:58:06.294953 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.294588    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" containerName="manager"
Jul 23 21:58:06.297524 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.297232    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:06.299624 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.299591    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:06.300405 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.300383    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:06.303477 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.303454    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"]
Jul 23 21:58:06.315939 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.315907    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"]
Jul 23 21:58:06.353282 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.353230    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:06.355194 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:06.355165    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:06.455020 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.454978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l585l\" (UniqueName: \"kubernetes.io/projected/09545f90-244e-4b34-a08f-f5ea41d4123f-kube-api-access-l585l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf47s\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.455219 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.455036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvx6d\" (UniqueName: \"kubernetes.io/projected/8bca412e-f8b7-40bc-bd2b-2343f4da7bbf-kube-api-access-lvx6d\") pod \"limitador-operator-controller-manager-859d7b78bd-2ncd7\" (UID: \"8bca412e-f8b7-40bc-bd2b-2343f4da7bbf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:06.455219 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.455072    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09545f90-244e-4b34-a08f-f5ea41d4123f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf47s\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.556674 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.556581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l585l\" (UniqueName: \"kubernetes.io/projected/09545f90-244e-4b34-a08f-f5ea41d4123f-kube-api-access-l585l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf47s\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.556846 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.556625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvx6d\" (UniqueName: \"kubernetes.io/projected/8bca412e-f8b7-40bc-bd2b-2343f4da7bbf-kube-api-access-lvx6d\") pod \"limitador-operator-controller-manager-859d7b78bd-2ncd7\" (UID: \"8bca412e-f8b7-40bc-bd2b-2343f4da7bbf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:06.556846 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.556705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09545f90-244e-4b34-a08f-f5ea41d4123f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf47s\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.557159 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.557134    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09545f90-244e-4b34-a08f-f5ea41d4123f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf47s\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.567371 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.567337    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l585l\" (UniqueName: \"kubernetes.io/projected/09545f90-244e-4b34-a08f-f5ea41d4123f-kube-api-access-l585l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf47s\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.567524 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.567398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvx6d\" (UniqueName: \"kubernetes.io/projected/8bca412e-f8b7-40bc-bd2b-2343f4da7bbf-kube-api-access-lvx6d\") pod \"limitador-operator-controller-manager-859d7b78bd-2ncd7\" (UID: \"8bca412e-f8b7-40bc-bd2b-2343f4da7bbf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:06.647078 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.647044    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:06.655093 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:06.655057    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:07.963225 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:07.963188    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"]
Jul 23 21:58:07.986737 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:07.986703    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"]
Jul 23 21:58:07.986902 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:07.986812    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.011663 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:08.011608    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:08.013347 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:08.013315    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:08.173945 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.173895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6xwr\" (UniqueName: \"kubernetes.io/projected/956713b4-2cd5-409e-bc9d-d28972e4d429-kube-api-access-b6xwr\") pod \"kuadrant-operator-controller-manager-86878c769f-l947j\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.174151 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.174008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/956713b4-2cd5-409e-bc9d-d28972e4d429-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l947j\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.227326 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:08.227238    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:08.229182 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:08.229144    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:08.274708 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.274665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/956713b4-2cd5-409e-bc9d-d28972e4d429-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l947j\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.274889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.274787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6xwr\" (UniqueName: \"kubernetes.io/projected/956713b4-2cd5-409e-bc9d-d28972e4d429-kube-api-access-b6xwr\") pod \"kuadrant-operator-controller-manager-86878c769f-l947j\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.275055 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.275032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/956713b4-2cd5-409e-bc9d-d28972e4d429-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l947j\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.289392 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.289364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6xwr\" (UniqueName: \"kubernetes.io/projected/956713b4-2cd5-409e-bc9d-d28972e4d429-kube-api-access-b6xwr\") pod \"kuadrant-operator-controller-manager-86878c769f-l947j\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:08.301259 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:08.301232    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:10.438200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:10.438134    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:10.549371 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:10.549330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:12.405487 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:12.405450    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9k6h"
Jul 23 21:58:15.217471 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:15.217423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:15.339021 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:15.338976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:19.168414 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.168350    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:19.172423 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.172394    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:19.173400 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.173369    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:19.175545 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.175498    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:19.178616 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.178528    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:19.186588 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.186560    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"]
Jul 23 21:58:19.189702 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:58:19.189660    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956713b4_2cd5_409e_bc9d_d28972e4d429.slice/crio-255f52eed2c0c9ff784897068745ae0dac8cfe0682f3d70fc91fe7197bf5f06d WatchSource:0}: Error finding container 255f52eed2c0c9ff784897068745ae0dac8cfe0682f3d70fc91fe7197bf5f06d: Status 404 returned error can't find the container with id 255f52eed2c0c9ff784897068745ae0dac8cfe0682f3d70fc91fe7197bf5f06d
Jul 23 21:58:19.225621 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.225433    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:19.249984 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.249950    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6986648f48-f86gf"]
Jul 23 21:58:19.253528 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:58:19.253490    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da43441_b5b6_4837_9adf_4a6407d7c859.slice/crio-bb22ad05b9709a0915075248663ceb6ce902a3a13c544f0850952df8666c9659 WatchSource:0}: Error finding container bb22ad05b9709a0915075248663ceb6ce902a3a13c544f0850952df8666c9659: Status 404 returned error can't find the container with id bb22ad05b9709a0915075248663ceb6ce902a3a13c544f0850952df8666c9659
Jul 23 21:58:19.276938 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.276907    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"]
Jul 23 21:58:19.280545 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.280519    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khhql\" (UniqueName: \"kubernetes.io/projected/1b3d5e35-b400-4e10-8931-81c8d79ecc00-kube-api-access-khhql\") pod \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") "
Jul 23 21:58:19.280662 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.280587    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b3d5e35-b400-4e10-8931-81c8d79ecc00-extensions-socket-volume\") pod \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\" (UID: \"1b3d5e35-b400-4e10-8931-81c8d79ecc00\") "
Jul 23 21:58:19.280662 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.280659    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5zhr\" (UniqueName: \"kubernetes.io/projected/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c-kube-api-access-n5zhr\") pod \"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c\" (UID: \"b9dd2ea7-ebe7-400a-97c3-aa89398bd81c\") "
Jul 23 21:58:19.281173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.281144    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d5e35-b400-4e10-8931-81c8d79ecc00-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1b3d5e35-b400-4e10-8931-81c8d79ecc00" (UID: "1b3d5e35-b400-4e10-8931-81c8d79ecc00"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:58:19.282717 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.282689    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3d5e35-b400-4e10-8931-81c8d79ecc00-kube-api-access-khhql" (OuterVolumeSpecName: "kube-api-access-khhql") pod "1b3d5e35-b400-4e10-8931-81c8d79ecc00" (UID: "1b3d5e35-b400-4e10-8931-81c8d79ecc00"). InnerVolumeSpecName "kube-api-access-khhql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:58:19.282717 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.282702    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c-kube-api-access-n5zhr" (OuterVolumeSpecName: "kube-api-access-n5zhr") pod "b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" (UID: "b9dd2ea7-ebe7-400a-97c3-aa89398bd81c"). InnerVolumeSpecName "kube-api-access-n5zhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:58:19.288733 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:58:19.288704    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09545f90_244e_4b34_a08f_f5ea41d4123f.slice/crio-0d0105f0480ceecb30e6a248e6eaf1b5e55291e179430d8848473f72c0833bdc WatchSource:0}: Error finding container 0d0105f0480ceecb30e6a248e6eaf1b5e55291e179430d8848473f72c0833bdc: Status 404 returned error can't find the container with id 0d0105f0480ceecb30e6a248e6eaf1b5e55291e179430d8848473f72c0833bdc
Jul 23 21:58:19.305739 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.305614    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"]
Jul 23 21:58:19.309425 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:58:19.309392    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bca412e_f8b7_40bc_bd2b_2343f4da7bbf.slice/crio-95207e8ee70638fa5c68725295d15bc1762523b4e4fad61c31dd618391e07d8a WatchSource:0}: Error finding container 95207e8ee70638fa5c68725295d15bc1762523b4e4fad61c31dd618391e07d8a: Status 404 returned error can't find the container with id 95207e8ee70638fa5c68725295d15bc1762523b4e4fad61c31dd618391e07d8a
Jul 23 21:58:19.368652 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.368605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:19.381974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.381943    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khhql\" (UniqueName: \"kubernetes.io/projected/1b3d5e35-b400-4e10-8931-81c8d79ecc00-kube-api-access-khhql\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:19.381974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.381969    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b3d5e35-b400-4e10-8931-81c8d79ecc00-extensions-socket-volume\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:19.382132 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.381978    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5zhr\" (UniqueName: \"kubernetes.io/projected/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c-kube-api-access-n5zhr\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:19.517667 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.517550    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j" event={"ID":"956713b4-2cd5-409e-bc9d-d28972e4d429","Type":"ContainerStarted","Data":"d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee"}
Jul 23 21:58:19.517667 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.517615    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j" event={"ID":"956713b4-2cd5-409e-bc9d-d28972e4d429","Type":"ContainerStarted","Data":"255f52eed2c0c9ff784897068745ae0dac8cfe0682f3d70fc91fe7197bf5f06d"}
Jul 23 21:58:19.517886 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.517766    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:19.519094 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.519066    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6" event={"ID":"bfd8d6b4-d360-43d4-b11f-793a9d63b9a3","Type":"ContainerStarted","Data":"d9a8bbe6b7ed1049ee367b9a18adfb5bb6308387897e2110362941118d7938dc"}
Jul 23 21:58:19.519760 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.519731    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:19.520422 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.520401    2575 generic.go:356] "Generic (PLEG): container finished" podID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" containerID="18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237" exitCode=0
Jul 23 21:58:19.520519 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.520424    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:19.520519 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.520484    2575 scope.go:117] "RemoveContainer" containerID="18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237"
Jul 23 21:58:19.521889 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.521865    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" containerID="26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430" exitCode=0
Jul 23 21:58:19.522001 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.521904    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:19.522105 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.522069    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:19.523850 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.523824    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7" event={"ID":"8bca412e-f8b7-40bc-bd2b-2343f4da7bbf","Type":"ContainerStarted","Data":"726a2a807e672e6f8eaa7c53cee73086261baa078f7a2aafd5afdd2f0084fd0b"}
Jul 23 21:58:19.523959 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.523869    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7" event={"ID":"8bca412e-f8b7-40bc-bd2b-2343f4da7bbf","Type":"ContainerStarted","Data":"95207e8ee70638fa5c68725295d15bc1762523b4e4fad61c31dd618391e07d8a"}
Jul 23 21:58:19.523959 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.523899    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:19.525981 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.525623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s" event={"ID":"09545f90-244e-4b34-a08f-f5ea41d4123f","Type":"ContainerStarted","Data":"d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763"}
Jul 23 21:58:19.525981 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.525761    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:19.525981 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.525786    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s" event={"ID":"09545f90-244e-4b34-a08f-f5ea41d4123f","Type":"ContainerStarted","Data":"0d0105f0480ceecb30e6a248e6eaf1b5e55291e179430d8848473f72c0833bdc"}
Jul 23 21:58:19.528491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.528011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6986648f48-f86gf" event={"ID":"0da43441-b5b6-4837-9adf-4a6407d7c859","Type":"ContainerStarted","Data":"3276d0ddf53327dc5f1882a24a2079547a77383b1174b1e9f2d6defc2d949df5"}
Jul 23 21:58:19.528491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.528048    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6986648f48-f86gf" event={"ID":"0da43441-b5b6-4837-9adf-4a6407d7c859","Type":"ContainerStarted","Data":"bb22ad05b9709a0915075248663ceb6ce902a3a13c544f0850952df8666c9659"}
Jul 23 21:58:19.532319 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.532284    2575 scope.go:117] "RemoveContainer" containerID="18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237"
Jul 23 21:58:19.532624 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.532581    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237\": container with ID starting with 18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237 not found: ID does not exist" containerID="18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237"
Jul 23 21:58:19.532776 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.532617    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237"} err="failed to get container status \"18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237\": rpc error: code = NotFound desc = could not find container \"18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237\": container with ID starting with 18f078399a5a49bf321112b24886dfad3b8a8b686680c43343be3b7b14dc0237 not found: ID does not exist"
Jul 23 21:58:19.532776 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.532692    2575 scope.go:117] "RemoveContainer" containerID="26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430"
Jul 23 21:58:19.541795 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.541775    2575 scope.go:117] "RemoveContainer" containerID="26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430"
Jul 23 21:58:19.542091 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.542070    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430\": container with ID starting with 26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430 not found: ID does not exist" containerID="26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430"
Jul 23 21:58:19.542164 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.542100    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430"} err="failed to get container status \"26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430\": rpc error: code = NotFound desc = could not find container \"26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430\": container with ID starting with 26441c57cb6046431488f5af429b7e58c3f33a04d8d6215a7f954cf7d8178430 not found: ID does not exist"
Jul 23 21:58:19.554264 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.554105    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j" podStartSLOduration=12.554087699 podStartE2EDuration="12.554087699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:19.552328599 +0000 UTC m=+421.813328327" watchObservedRunningTime="2026-07-23 21:58:19.554087699 +0000 UTC m=+421.815087404"
Jul 23 21:58:19.581427 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.581371    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7" podStartSLOduration=13.58135616 podStartE2EDuration="13.58135616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:19.580581962 +0000 UTC m=+421.841581714" watchObservedRunningTime="2026-07-23 21:58:19.58135616 +0000 UTC m=+421.842355888"
Jul 23 21:58:19.603522 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.603391    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2ncs6" podStartSLOduration=1.889152699 podStartE2EDuration="24.603371347s" totalImagesPullingTime="22.714218648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:19.603367856 +0000 UTC m=+421.864367596" watchObservedRunningTime="2026-07-23 21:58:19.603371347 +0000 UTC m=+421.864371074"
Jul 23 21:58:19.605968 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.605937    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-82dw8\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-82dw8"
Jul 23 21:58:19.608419 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:19.608400    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tmdh7\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tmdh7"
Jul 23 21:58:19.635923 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.635850    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6986648f48-f86gf" podStartSLOduration=14.63582749 podStartE2EDuration="14.63582749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:19.634443915 +0000 UTC m=+421.895443670" watchObservedRunningTime="2026-07-23 21:58:19.63582749 +0000 UTC m=+421.896827219"
Jul 23 21:58:19.660012 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:19.659951    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s" podStartSLOduration=13.659934048 podStartE2EDuration="13.659934048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:19.654341473 +0000 UTC m=+421.915341202" watchObservedRunningTime="2026-07-23 21:58:19.659934048 +0000 UTC m=+421.920933840"
Jul 23 21:58:20.228186 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:20.228109    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b3d5e35-b400-4e10-8931-81c8d79ecc00" path="/var/lib/kubelet/pods/1b3d5e35-b400-4e10-8931-81c8d79ecc00/volumes"
Jul 23 21:58:20.228627 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:20.228609    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9dd2ea7-ebe7-400a-97c3-aa89398bd81c" path="/var/lib/kubelet/pods/b9dd2ea7-ebe7-400a-97c3-aa89398bd81c/volumes"
Jul 23 21:58:25.480518 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.480479    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:25.481003 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.480553    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:25.485389 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.485359    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:25.576885 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.576852    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6986648f48-f86gf"
Jul 23 21:58:25.625571 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.625523    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bcc799d49-tgdq4"]
Jul 23 21:58:25.727927 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.727892    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:25.837173 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.837087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:25.949706 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:25.949666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:26.574259 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:26.574227    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:26.631048 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:26.631013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:30.538121 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.538089    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:30.538572 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.538261    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 21:58:30.538572 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.538319    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2ncd7"
Jul 23 21:58:30.636386 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.636352    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"]
Jul 23 21:58:30.636651 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.636605    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s" podUID="09545f90-244e-4b34-a08f-f5ea41d4123f" containerName="manager" containerID="cri-o://d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763" gracePeriod=10
Jul 23 21:58:30.919239 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.919189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:30.930499 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.930472    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:30.985044 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.984961    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09545f90-244e-4b34-a08f-f5ea41d4123f-extensions-socket-volume\") pod \"09545f90-244e-4b34-a08f-f5ea41d4123f\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") "
Jul 23 21:58:30.985044 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.985016    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l585l\" (UniqueName: \"kubernetes.io/projected/09545f90-244e-4b34-a08f-f5ea41d4123f-kube-api-access-l585l\") pod \"09545f90-244e-4b34-a08f-f5ea41d4123f\" (UID: \"09545f90-244e-4b34-a08f-f5ea41d4123f\") "
Jul 23 21:58:30.985360 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.985336    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09545f90-244e-4b34-a08f-f5ea41d4123f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "09545f90-244e-4b34-a08f-f5ea41d4123f" (UID: "09545f90-244e-4b34-a08f-f5ea41d4123f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 21:58:30.987069 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:30.987042    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09545f90-244e-4b34-a08f-f5ea41d4123f-kube-api-access-l585l" (OuterVolumeSpecName: "kube-api-access-l585l") pod "09545f90-244e-4b34-a08f-f5ea41d4123f" (UID: "09545f90-244e-4b34-a08f-f5ea41d4123f"). InnerVolumeSpecName "kube-api-access-l585l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:58:31.049612 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.049574    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:31.085918 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.085840    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09545f90-244e-4b34-a08f-f5ea41d4123f-extensions-socket-volume\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:31.085918 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.085879    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l585l\" (UniqueName: \"kubernetes.io/projected/09545f90-244e-4b34-a08f-f5ea41d4123f-kube-api-access-l585l\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:31.608750 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.608709    2575 generic.go:356] "Generic (PLEG): container finished" podID="09545f90-244e-4b34-a08f-f5ea41d4123f" containerID="d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763" exitCode=0
Jul 23 21:58:31.609200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.608758    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"
Jul 23 21:58:31.609200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.608757    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s" event={"ID":"09545f90-244e-4b34-a08f-f5ea41d4123f","Type":"ContainerDied","Data":"d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763"}
Jul 23 21:58:31.609200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.608904    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s" event={"ID":"09545f90-244e-4b34-a08f-f5ea41d4123f","Type":"ContainerDied","Data":"0d0105f0480ceecb30e6a248e6eaf1b5e55291e179430d8848473f72c0833bdc"}
Jul 23 21:58:31.609200 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.608946    2575 scope.go:117] "RemoveContainer" containerID="d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763"
Jul 23 21:58:31.618024 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.618008    2575 scope.go:117] "RemoveContainer" containerID="d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763"
Jul 23 21:58:31.618279 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:58:31.618262    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763\": container with ID starting with d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763 not found: ID does not exist" containerID="d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763"
Jul 23 21:58:31.618317 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.618288    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763"} err="failed to get container status \"d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763\": rpc error: code = NotFound desc = could not find container \"d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763\": container with ID starting with d6765c0cb9e98abaf12d21dd18ec47385ad1e35f2895cb825e488b87cf349763 not found: ID does not exist"
Jul 23 21:58:31.630707 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.630683    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"]
Jul 23 21:58:31.645642 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:31.645608    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf47s"]
Jul 23 21:58:32.227201 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:32.227169    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09545f90-244e-4b34-a08f-f5ea41d4123f" path="/var/lib/kubelet/pods/09545f90-244e-4b34-a08f-f5ea41d4123f/volumes"
Jul 23 21:58:35.587159 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:35.587123    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:35.685077 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:35.685042    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:47.134430 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:47.134344    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:47.322962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:47.322923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:48.838186 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.838143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:48.839314 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.839287    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:58:48.839725 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.839708    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09545f90-244e-4b34-a08f-f5ea41d4123f" containerName="manager"
Jul 23 21:58:48.839725 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.839724    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="09545f90-244e-4b34-a08f-f5ea41d4123f" containerName="manager"
Jul 23 21:58:48.839856 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.839790    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="09545f90-244e-4b34-a08f-f5ea41d4123f" containerName="manager"
Jul 23 21:58:48.894371 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.894336    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:58:48.894535 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.894446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:48.897093 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.897069    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 21:58:48.933122 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.933089    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:58:48.943232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:48.943185    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:49.042680 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.042649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef224bc4-73c3-426c-951f-5988651558a2-config-file\") pod \"limitador-limitador-64c94fd64f-j5786\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.042866 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.042699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbbd9\" (UniqueName: \"kubernetes.io/projected/ef224bc4-73c3-426c-951f-5988651558a2-kube-api-access-nbbd9\") pod \"limitador-limitador-64c94fd64f-j5786\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.046314 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.046276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:49.143608 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.143570    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef224bc4-73c3-426c-951f-5988651558a2-config-file\") pod \"limitador-limitador-64c94fd64f-j5786\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.143834 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.143622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbbd9\" (UniqueName: \"kubernetes.io/projected/ef224bc4-73c3-426c-951f-5988651558a2-kube-api-access-nbbd9\") pod \"limitador-limitador-64c94fd64f-j5786\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.144352 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.144326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef224bc4-73c3-426c-951f-5988651558a2-config-file\") pod \"limitador-limitador-64c94fd64f-j5786\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.151640 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.151604    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbbd9\" (UniqueName: \"kubernetes.io/projected/ef224bc4-73c3-426c-951f-5988651558a2-kube-api-access-nbbd9\") pod \"limitador-limitador-64c94fd64f-j5786\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.205444 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.205408    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:49.350245 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.350220    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:58:49.351868 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:58:49.351840    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef224bc4_73c3_426c_951f_5988651558a2.slice/crio-a65ccd98b5cf82763c2fc46823a6b3fe9bcbe3dc93384eea5d08ead10cb36e0c WatchSource:0}: Error finding container a65ccd98b5cf82763c2fc46823a6b3fe9bcbe3dc93384eea5d08ead10cb36e0c: Status 404 returned error can't find the container with id a65ccd98b5cf82763c2fc46823a6b3fe9bcbe3dc93384eea5d08ead10cb36e0c
Jul 23 21:58:49.679896 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.679855    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:49.723472 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.723438    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786" event={"ID":"ef224bc4-73c3-426c-951f-5988651558a2","Type":"ContainerStarted","Data":"a65ccd98b5cf82763c2fc46823a6b3fe9bcbe3dc93384eea5d08ead10cb36e0c"}
Jul 23 21:58:49.782479 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.782444    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:49.795528 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.795494    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-t5jfm"]
Jul 23 21:58:49.800256 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.800236    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:58:49.802599 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.802576    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mh59l\""
Jul 23 21:58:49.809332 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.809305    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t5jfm"]
Jul 23 21:58:49.950175 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:49.950105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5krn\" (UniqueName: \"kubernetes.io/projected/11f96823-6e2c-42b4-95a3-9d28c28339f3-kube-api-access-g5krn\") pod \"authorino-75d556bd77-t5jfm\" (UID: \"11f96823-6e2c-42b4-95a3-9d28c28339f3\") " pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:58:50.051067 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:50.051030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5krn\" (UniqueName: \"kubernetes.io/projected/11f96823-6e2c-42b4-95a3-9d28c28339f3-kube-api-access-g5krn\") pod \"authorino-75d556bd77-t5jfm\" (UID: \"11f96823-6e2c-42b4-95a3-9d28c28339f3\") " pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:58:50.061147 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:50.061112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5krn\" (UniqueName: \"kubernetes.io/projected/11f96823-6e2c-42b4-95a3-9d28c28339f3-kube-api-access-g5krn\") pod \"authorino-75d556bd77-t5jfm\" (UID: \"11f96823-6e2c-42b4-95a3-9d28c28339f3\") " pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:58:50.109980 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:50.109943    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:58:50.290270 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:50.290233    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t5jfm"]
Jul 23 21:58:50.293125 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:58:50.293091    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11f96823_6e2c_42b4_95a3_9d28c28339f3.slice/crio-665de7a39b9e8f788befd93d04d009f50c09ba2b6ab70e20c447fa37797f4597 WatchSource:0}: Error finding container 665de7a39b9e8f788befd93d04d009f50c09ba2b6ab70e20c447fa37797f4597: Status 404 returned error can't find the container with id 665de7a39b9e8f788befd93d04d009f50c09ba2b6ab70e20c447fa37797f4597
Jul 23 21:58:50.649858 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:50.649805    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bcc799d49-tgdq4" podUID="93ab960f-3f43-4966-a336-043f98b88d9f" containerName="console" containerID="cri-o://aa28ca03033b955a13b9a3479e99112f366c93399c9926bb2beeef756766ba7e" gracePeriod=15
Jul 23 21:58:50.731505 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:50.731457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t5jfm" event={"ID":"11f96823-6e2c-42b4-95a3-9d28c28339f3","Type":"ContainerStarted","Data":"665de7a39b9e8f788befd93d04d009f50c09ba2b6ab70e20c447fa37797f4597"}
Jul 23 21:58:51.742529 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:51.742493    2575 generic.go:356] "Generic (PLEG): container finished" podID="93ab960f-3f43-4966-a336-043f98b88d9f" containerID="aa28ca03033b955a13b9a3479e99112f366c93399c9926bb2beeef756766ba7e" exitCode=0
Jul 23 21:58:51.742967 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:51.742552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc799d49-tgdq4" event={"ID":"93ab960f-3f43-4966-a336-043f98b88d9f","Type":"ContainerDied","Data":"aa28ca03033b955a13b9a3479e99112f366c93399c9926bb2beeef756766ba7e"}
Jul 23 21:58:51.812988 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:51.812927    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:51.863474 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:51.863426    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:58:52.656485 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.656456    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:58:52.750284 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.750256    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc799d49-tgdq4"
Jul 23 21:58:52.750756 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.750313    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc799d49-tgdq4" event={"ID":"93ab960f-3f43-4966-a336-043f98b88d9f","Type":"ContainerDied","Data":"b0cd7c802c71ef6930ba7af39d5fbdbd4e69ced83a464ac1c0bccab337c7f3ba"}
Jul 23 21:58:52.750756 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.750377    2575 scope.go:117] "RemoveContainer" containerID="aa28ca03033b955a13b9a3479e99112f366c93399c9926bb2beeef756766ba7e"
Jul 23 21:58:52.780996 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.780965    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-oauth-config\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781016    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-serving-cert\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781059    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-console-config\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781096    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5sld\" (UniqueName: \"kubernetes.io/projected/93ab960f-3f43-4966-a336-043f98b88d9f-kube-api-access-z5sld\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781125    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-trusted-ca-bundle\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-service-ca\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781181 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781172    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-oauth-serving-cert\") pod \"93ab960f-3f43-4966-a336-043f98b88d9f\" (UID: \"93ab960f-3f43-4966-a336-043f98b88d9f\") "
Jul 23 21:58:52.781534 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781505    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-console-config" (OuterVolumeSpecName: "console-config") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:58:52.781753 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781582    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:58:52.781856 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.781753    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-service-ca" (OuterVolumeSpecName: "service-ca") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:58:52.782097 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.782075    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:58:52.783146 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.783117    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:58:52.783735 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.783713    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 21:58:52.783834 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.783728    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ab960f-3f43-4966-a336-043f98b88d9f-kube-api-access-z5sld" (OuterVolumeSpecName: "kube-api-access-z5sld") pod "93ab960f-3f43-4966-a336-043f98b88d9f" (UID: "93ab960f-3f43-4966-a336-043f98b88d9f"). InnerVolumeSpecName "kube-api-access-z5sld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:58:52.882153 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882108    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-serving-cert\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:52.882153 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882154    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-console-config\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:52.882377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882170    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5sld\" (UniqueName: \"kubernetes.io/projected/93ab960f-3f43-4966-a336-043f98b88d9f-kube-api-access-z5sld\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:52.882377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882185    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-trusted-ca-bundle\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:52.882377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882199    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:52.882377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882212    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93ab960f-3f43-4966-a336-043f98b88d9f-oauth-serving-cert\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:52.882377 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:52.882225    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93ab960f-3f43-4966-a336-043f98b88d9f-console-oauth-config\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:58:53.078546 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:53.078503    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bcc799d49-tgdq4"]
Jul 23 21:58:53.082559 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:53.082531    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bcc799d49-tgdq4"]
Jul 23 21:58:53.200033 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:53.199993    2575 patch_prober.go:28] interesting pod/console-5bcc799d49-tgdq4 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.23:8443/health\": context deadline exceeded" start-of-body=
Jul 23 21:58:53.200214 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:53.200062    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-5bcc799d49-tgdq4" podUID="93ab960f-3f43-4966-a336-043f98b88d9f" containerName="console" probeResult="failure" output="Get \"https://10.132.0.23:8443/health\": context deadline exceeded"
Jul 23 21:58:54.228232 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:54.228189    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ab960f-3f43-4966-a336-043f98b88d9f" path="/var/lib/kubelet/pods/93ab960f-3f43-4966-a336-043f98b88d9f/volumes"
Jul 23 21:58:54.765414 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:54.765379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786" event={"ID":"ef224bc4-73c3-426c-951f-5988651558a2","Type":"ContainerStarted","Data":"8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1"}
Jul 23 21:58:54.765610 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:54.765441    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:58:54.766917 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:54.766893    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t5jfm" event={"ID":"11f96823-6e2c-42b4-95a3-9d28c28339f3","Type":"ContainerStarted","Data":"7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0"}
Jul 23 21:58:54.783196 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:54.783154    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786" podStartSLOduration=2.035919322 podStartE2EDuration="6.783142091s" totalImagesPullingTime="4.747222769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:54.780669022 +0000 UTC m=+457.041668774" watchObservedRunningTime="2026-07-23 21:58:54.783142091 +0000 UTC m=+457.044141819"
Jul 23 21:58:54.793326 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:58:54.793279    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-t5jfm" podStartSLOduration=1.986021913 podStartE2EDuration="5.793265777s" totalImagesPullingTime="3.807243864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:54.793163052 +0000 UTC m=+457.054162793" watchObservedRunningTime="2026-07-23 21:58:54.793265777 +0000 UTC m=+457.054265504"
Jul 23 21:59:03.823452 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:03.823418    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:59:03.824053 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:03.823687    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786" podUID="ef224bc4-73c3-426c-951f-5988651558a2" containerName="limitador" containerID="cri-o://8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1" gracePeriod=30
Jul 23 21:59:03.824435 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:03.824288    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:59:04.377488 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.377459    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:59:04.477491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.477391    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbbd9\" (UniqueName: \"kubernetes.io/projected/ef224bc4-73c3-426c-951f-5988651558a2-kube-api-access-nbbd9\") pod \"ef224bc4-73c3-426c-951f-5988651558a2\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") "
Jul 23 21:59:04.477491 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.477454    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef224bc4-73c3-426c-951f-5988651558a2-config-file\") pod \"ef224bc4-73c3-426c-951f-5988651558a2\" (UID: \"ef224bc4-73c3-426c-951f-5988651558a2\") "
Jul 23 21:59:04.477851 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.477827    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef224bc4-73c3-426c-951f-5988651558a2-config-file" (OuterVolumeSpecName: "config-file") pod "ef224bc4-73c3-426c-951f-5988651558a2" (UID: "ef224bc4-73c3-426c-951f-5988651558a2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 21:59:04.479480 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.479455    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef224bc4-73c3-426c-951f-5988651558a2-kube-api-access-nbbd9" (OuterVolumeSpecName: "kube-api-access-nbbd9") pod "ef224bc4-73c3-426c-951f-5988651558a2" (UID: "ef224bc4-73c3-426c-951f-5988651558a2"). InnerVolumeSpecName "kube-api-access-nbbd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:59:04.578878 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.578838    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef224bc4-73c3-426c-951f-5988651558a2-config-file\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:59:04.578878 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.578872    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbbd9\" (UniqueName: \"kubernetes.io/projected/ef224bc4-73c3-426c-951f-5988651558a2-kube-api-access-nbbd9\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:59:04.828574 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.828477    2575 generic.go:356] "Generic (PLEG): container finished" podID="ef224bc4-73c3-426c-951f-5988651558a2" containerID="8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1" exitCode=0
Jul 23 21:59:04.828574 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.828530    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786"
Jul 23 21:59:04.828574 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.828564    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786" event={"ID":"ef224bc4-73c3-426c-951f-5988651558a2","Type":"ContainerDied","Data":"8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1"}
Jul 23 21:59:04.829134 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.828616    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j5786" event={"ID":"ef224bc4-73c3-426c-951f-5988651558a2","Type":"ContainerDied","Data":"a65ccd98b5cf82763c2fc46823a6b3fe9bcbe3dc93384eea5d08ead10cb36e0c"}
Jul 23 21:59:04.829134 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.828663    2575 scope.go:117] "RemoveContainer" containerID="8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1"
Jul 23 21:59:04.837896 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.837877    2575 scope.go:117] "RemoveContainer" containerID="8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1"
Jul 23 21:59:04.838161 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:59:04.838140    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1\": container with ID starting with 8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1 not found: ID does not exist" containerID="8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1"
Jul 23 21:59:04.838212 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.838170    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1"} err="failed to get container status \"8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1\": rpc error: code = NotFound desc = could not find container \"8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1\": container with ID starting with 8696c52cc90d922dccf0a892bdfc5a2160254b8072e372853630d906811713b1 not found: ID does not exist"
Jul 23 21:59:04.850193 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.850166    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:59:04.852334 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.852312    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j5786"]
Jul 23 21:59:04.992511 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:04.992483    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:05.095237 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.095206    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:05.471806 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.471710    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"]
Jul 23 21:59:05.472327 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.472305    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ab960f-3f43-4966-a336-043f98b88d9f" containerName="console"
Jul 23 21:59:05.472411 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.472331    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ab960f-3f43-4966-a336-043f98b88d9f" containerName="console"
Jul 23 21:59:05.472411 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.472352    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef224bc4-73c3-426c-951f-5988651558a2" containerName="limitador"
Jul 23 21:59:05.472411 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.472360    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef224bc4-73c3-426c-951f-5988651558a2" containerName="limitador"
Jul 23 21:59:05.472562 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.472456    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ab960f-3f43-4966-a336-043f98b88d9f" containerName="console"
Jul 23 21:59:05.472562 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.472474    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef224bc4-73c3-426c-951f-5988651558a2" containerName="limitador"
Jul 23 21:59:05.477469 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.477446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.480548 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.480515    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 21:59:05.480741 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.480625    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 23 21:59:05.480902 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.480874    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 21:59:05.481379 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.481359    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bbrk5\""
Jul 23 21:59:05.482392 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.482358    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"]
Jul 23 21:59:05.506770 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.506727    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:05.586420 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.586377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bf3375f2-cce1-4df0-8e1c-94f7e11115bd-data\") pod \"postgres-69fb4594cf-pl4k2\" (UID: \"bf3375f2-cce1-4df0-8e1c-94f7e11115bd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.586586 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.586428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x92n\" (UniqueName: \"kubernetes.io/projected/bf3375f2-cce1-4df0-8e1c-94f7e11115bd-kube-api-access-5x92n\") pod \"postgres-69fb4594cf-pl4k2\" (UID: \"bf3375f2-cce1-4df0-8e1c-94f7e11115bd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.687034 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.686993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bf3375f2-cce1-4df0-8e1c-94f7e11115bd-data\") pod \"postgres-69fb4594cf-pl4k2\" (UID: \"bf3375f2-cce1-4df0-8e1c-94f7e11115bd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.687210 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.687042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x92n\" (UniqueName: \"kubernetes.io/projected/bf3375f2-cce1-4df0-8e1c-94f7e11115bd-kube-api-access-5x92n\") pod \"postgres-69fb4594cf-pl4k2\" (UID: \"bf3375f2-cce1-4df0-8e1c-94f7e11115bd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.687426 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.687403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bf3375f2-cce1-4df0-8e1c-94f7e11115bd-data\") pod \"postgres-69fb4594cf-pl4k2\" (UID: \"bf3375f2-cce1-4df0-8e1c-94f7e11115bd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.696202 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.696168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x92n\" (UniqueName: \"kubernetes.io/projected/bf3375f2-cce1-4df0-8e1c-94f7e11115bd-kube-api-access-5x92n\") pod \"postgres-69fb4594cf-pl4k2\" (UID: \"bf3375f2-cce1-4df0-8e1c-94f7e11115bd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.791031 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.790944    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:05.928925 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:05.928901    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"]
Jul 23 21:59:05.930708 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:59:05.930683    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3375f2_cce1_4df0_8e1c_94f7e11115bd.slice/crio-c9f7a8f4bb4fa25b7d3c8a4289aaf164f4773b60955ca170c8b75b64b445854e WatchSource:0}: Error finding container c9f7a8f4bb4fa25b7d3c8a4289aaf164f4773b60955ca170c8b75b64b445854e: Status 404 returned error can't find the container with id c9f7a8f4bb4fa25b7d3c8a4289aaf164f4773b60955ca170c8b75b64b445854e
Jul 23 21:59:06.226795 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:06.226760    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef224bc4-73c3-426c-951f-5988651558a2" path="/var/lib/kubelet/pods/ef224bc4-73c3-426c-951f-5988651558a2/volumes"
Jul 23 21:59:06.854124 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:06.854067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2" event={"ID":"bf3375f2-cce1-4df0-8e1c-94f7e11115bd","Type":"ContainerStarted","Data":"c9f7a8f4bb4fa25b7d3c8a4289aaf164f4773b60955ca170c8b75b64b445854e"}
Jul 23 21:59:07.163265 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:07.163138    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:10.881586 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:10.881551    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2" event={"ID":"bf3375f2-cce1-4df0-8e1c-94f7e11115bd","Type":"ContainerStarted","Data":"2482e9ac9e4e43f8790616124f3d2dc477d18ec47fc77b53048035300ca4018a"}
Jul 23 21:59:10.881970 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:10.881612    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:10.897740 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:10.897691    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2" podStartSLOduration=1.106270279 podStartE2EDuration="5.897670648s" totalImagesPullingTime="4.791400369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:10.896098367 +0000 UTC m=+473.157098113" watchObservedRunningTime="2026-07-23 21:59:10.897670648 +0000 UTC m=+473.158670377"
Jul 23 21:59:16.922144 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:16.922113    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pl4k2"
Jul 23 21:59:17.242996 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:17.242903    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:17.545549 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:17.545466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:17.970847 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:17.970810    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58c8d7c7bb-p6djq"]
Jul 23 21:59:17.974933 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:17.974911    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:17.977122 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:17.977106    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 23 21:59:17.981853 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:17.981817    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58c8d7c7bb-p6djq"]
Jul 23 21:59:18.101769 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.101698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5qh\" (UniqueName: \"kubernetes.io/projected/0b59fba4-932a-46cb-8aff-7e79481f4bfa-kube-api-access-xt5qh\") pod \"authorino-58c8d7c7bb-p6djq\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") " pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.101939 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.101798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b59fba4-932a-46cb-8aff-7e79481f4bfa-tls-cert\") pod \"authorino-58c8d7c7bb-p6djq\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") " pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.203098 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.203067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b59fba4-932a-46cb-8aff-7e79481f4bfa-tls-cert\") pod \"authorino-58c8d7c7bb-p6djq\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") " pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.203262 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.203162    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5qh\" (UniqueName: \"kubernetes.io/projected/0b59fba4-932a-46cb-8aff-7e79481f4bfa-kube-api-access-xt5qh\") pod \"authorino-58c8d7c7bb-p6djq\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") " pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.208140 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.208116    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 23 21:59:18.216244 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.216217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b59fba4-932a-46cb-8aff-7e79481f4bfa-tls-cert\") pod \"authorino-58c8d7c7bb-p6djq\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") " pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.220962 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.220894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5qh\" (UniqueName: \"kubernetes.io/projected/0b59fba4-932a-46cb-8aff-7e79481f4bfa-kube-api-access-xt5qh\") pod \"authorino-58c8d7c7bb-p6djq\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") " pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.285308 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.285277    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 21:59:18.441930 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.441906    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58c8d7c7bb-p6djq"]
Jul 23 21:59:18.444148 ip-10-0-143-223 kubenswrapper[2575]: W0723 21:59:18.444121    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b59fba4_932a_46cb_8aff_7e79481f4bfa.slice/crio-44f73cb7f8c39c9b27a8b5c368c71ca5cfc680e78feb43f3107ea5e4c4f96899 WatchSource:0}: Error finding container 44f73cb7f8c39c9b27a8b5c368c71ca5cfc680e78feb43f3107ea5e4c4f96899: Status 404 returned error can't find the container with id 44f73cb7f8c39c9b27a8b5c368c71ca5cfc680e78feb43f3107ea5e4c4f96899
Jul 23 21:59:18.927740 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:18.927701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq" event={"ID":"0b59fba4-932a-46cb-8aff-7e79481f4bfa","Type":"ContainerStarted","Data":"44f73cb7f8c39c9b27a8b5c368c71ca5cfc680e78feb43f3107ea5e4c4f96899"}
Jul 23 21:59:19.934170 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:19.934137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq" event={"ID":"0b59fba4-932a-46cb-8aff-7e79481f4bfa","Type":"ContainerStarted","Data":"66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d"}
Jul 23 21:59:19.952844 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:19.952789    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq" podStartSLOduration=2.355287845 podStartE2EDuration="2.95277111s" totalImagesPullingTime="597.483265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:19.95129404 +0000 UTC m=+482.212293794" watchObservedRunningTime="2026-07-23 21:59:19.95277111 +0000 UTC m=+482.213770839"
Jul 23 21:59:19.982300 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:19.982264    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t5jfm"]
Jul 23 21:59:19.982510 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:19.982483    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-t5jfm" podUID="11f96823-6e2c-42b4-95a3-9d28c28339f3" containerName="authorino" containerID="cri-o://7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0" gracePeriod=30
Jul 23 21:59:20.278709 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.278678    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:59:20.321758 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.321720    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5krn\" (UniqueName: \"kubernetes.io/projected/11f96823-6e2c-42b4-95a3-9d28c28339f3-kube-api-access-g5krn\") pod \"11f96823-6e2c-42b4-95a3-9d28c28339f3\" (UID: \"11f96823-6e2c-42b4-95a3-9d28c28339f3\") "
Jul 23 21:59:20.323807 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.323771    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f96823-6e2c-42b4-95a3-9d28c28339f3-kube-api-access-g5krn" (OuterVolumeSpecName: "kube-api-access-g5krn") pod "11f96823-6e2c-42b4-95a3-9d28c28339f3" (UID: "11f96823-6e2c-42b4-95a3-9d28c28339f3"). InnerVolumeSpecName "kube-api-access-g5krn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 21:59:20.423202 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.423166    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5krn\" (UniqueName: \"kubernetes.io/projected/11f96823-6e2c-42b4-95a3-9d28c28339f3-kube-api-access-g5krn\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 21:59:20.941130 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.941089    2575 generic.go:356] "Generic (PLEG): container finished" podID="11f96823-6e2c-42b4-95a3-9d28c28339f3" containerID="7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0" exitCode=0
Jul 23 21:59:20.941130 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.941132    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t5jfm"
Jul 23 21:59:20.941654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.941165    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t5jfm" event={"ID":"11f96823-6e2c-42b4-95a3-9d28c28339f3","Type":"ContainerDied","Data":"7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0"}
Jul 23 21:59:20.941654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.941202    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t5jfm" event={"ID":"11f96823-6e2c-42b4-95a3-9d28c28339f3","Type":"ContainerDied","Data":"665de7a39b9e8f788befd93d04d009f50c09ba2b6ab70e20c447fa37797f4597"}
Jul 23 21:59:20.941654 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.941224    2575 scope.go:117] "RemoveContainer" containerID="7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0"
Jul 23 21:59:20.950612 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.950594    2575 scope.go:117] "RemoveContainer" containerID="7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0"
Jul 23 21:59:20.950880 ip-10-0-143-223 kubenswrapper[2575]: E0723 21:59:20.950861    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0\": container with ID starting with 7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0 not found: ID does not exist" containerID="7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0"
Jul 23 21:59:20.950921 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.950890    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0"} err="failed to get container status \"7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0\": rpc error: code = NotFound desc = could not find container \"7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0\": container with ID starting with 7f46763c266d684f8f4fa6aec491df806f40fd66218fb5aec3ff61a5402775c0 not found: ID does not exist"
Jul 23 21:59:20.964136 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.964108    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t5jfm"]
Jul 23 21:59:20.968730 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:20.968703    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t5jfm"]
Jul 23 21:59:22.227640 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:22.227606    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f96823-6e2c-42b4-95a3-9d28c28339f3" path="/var/lib/kubelet/pods/11f96823-6e2c-42b4-95a3-9d28c28339f3/volumes"
Jul 23 21:59:24.898705 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:24.898663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:25.076074 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:25.076019    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:25.176877 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:25.176791    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:25.250246 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:25.250216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:28.169383 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:28.169336    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:28.259875 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:28.259832    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:47.838115 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:47.838081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:47.944236 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:47.944187    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:58.145509 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:58.145467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:58.365349 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:58.365312    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:58.664939 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:58.664903    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:58.797746 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:58.797705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:59.278725 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.278687    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:59.416048 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.416011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 21:59:59.585423 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.585342    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-rspp6"]
Jul 23 21:59:59.585739 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.585725    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f96823-6e2c-42b4-95a3-9d28c28339f3" containerName="authorino"
Jul 23 21:59:59.585818 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.585740    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f96823-6e2c-42b4-95a3-9d28c28339f3" containerName="authorino"
Jul 23 21:59:59.585865 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.585821    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="11f96823-6e2c-42b4-95a3-9d28c28339f3" containerName="authorino"
Jul 23 21:59:59.589079 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.589059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.594845 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.594817    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-h6b7q\""
Jul 23 21:59:59.594974 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.594955    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 23 21:59:59.619127 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.619094    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-rspp6"]
Jul 23 21:59:59.657624 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.657585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cbjw\" (UniqueName: \"kubernetes.io/projected/5f5e8d22-53ca-4590-8af0-ccc1f1dfb503-kube-api-access-9cbjw\") pod \"payload-processing-5bf47cd988-rspp6\" (UID: \"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503\") " pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.657624 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.657623    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5e8d22-53ca-4590-8af0-ccc1f1dfb503-plugins-config-volume\") pod \"payload-processing-5bf47cd988-rspp6\" (UID: \"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503\") " pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.759056 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.759019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cbjw\" (UniqueName: \"kubernetes.io/projected/5f5e8d22-53ca-4590-8af0-ccc1f1dfb503-kube-api-access-9cbjw\") pod \"payload-processing-5bf47cd988-rspp6\" (UID: \"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503\") " pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.759056 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.759058    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5e8d22-53ca-4590-8af0-ccc1f1dfb503-plugins-config-volume\") pod \"payload-processing-5bf47cd988-rspp6\" (UID: \"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503\") " pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.759800 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.759781    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f5e8d22-53ca-4590-8af0-ccc1f1dfb503-plugins-config-volume\") pod \"payload-processing-5bf47cd988-rspp6\" (UID: \"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503\") " pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.769319 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.769293    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cbjw\" (UniqueName: \"kubernetes.io/projected/5f5e8d22-53ca-4590-8af0-ccc1f1dfb503-kube-api-access-9cbjw\") pod \"payload-processing-5bf47cd988-rspp6\" (UID: \"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503\") " pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 21:59:59.899417 ip-10-0-143-223 kubenswrapper[2575]: I0723 21:59:59.899376    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 22:00:00.148158 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.148124    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"]
Jul 23 22:00:00.152647 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.152579    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"
Jul 23 22:00:00.155066 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.155040    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:00:00.161442 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.161375    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"]
Jul 23 22:00:00.165268 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.165245    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"
Jul 23 22:00:00.250120 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.250092    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-rspp6"]
Jul 23 22:00:00.252468 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:00:00.252433    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5e8d22_53ca_4590_8af0_ccc1f1dfb503.slice/crio-92cc1d878fc1977f22199beb95f0376d9bb6e68b55a3868521a04aef10f60a8d WatchSource:0}: Error finding container 92cc1d878fc1977f22199beb95f0376d9bb6e68b55a3868521a04aef10f60a8d: Status 404 returned error can't find the container with id 92cc1d878fc1977f22199beb95f0376d9bb6e68b55a3868521a04aef10f60a8d
Jul 23 22:00:00.313381 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:00.313350    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"]
Jul 23 22:00:00.315441 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:00:00.315410    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfd2089_c419_4688_8c97_5821c0128683.slice/crio-0c5bc8892cef5994e8b29bf71a3963795a527e4c735d8862e5e5f83184d920b5 WatchSource:0}: Error finding container 0c5bc8892cef5994e8b29bf71a3963795a527e4c735d8862e5e5f83184d920b5: Status 404 returned error can't find the container with id 0c5bc8892cef5994e8b29bf71a3963795a527e4c735d8862e5e5f83184d920b5
Jul 23 22:00:01.170230 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:01.170190    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6" event={"ID":"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503","Type":"ContainerStarted","Data":"92cc1d878fc1977f22199beb95f0376d9bb6e68b55a3868521a04aef10f60a8d"}
Jul 23 22:00:01.171765 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:01.171726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" event={"ID":"2bfd2089-c419-4688-8c97-5821c0128683","Type":"ContainerStarted","Data":"0c5bc8892cef5994e8b29bf71a3963795a527e4c735d8862e5e5f83184d920b5"}
Jul 23 22:00:02.181341 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:02.181303    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6" event={"ID":"5f5e8d22-53ca-4590-8af0-ccc1f1dfb503","Type":"ContainerStarted","Data":"e55c8957bf47a14e25cdb805adfe06de54e973c261efdad44036da537994387c"}
Jul 23 22:00:02.181802 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:02.181370    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 22:00:02.205472 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:02.205417    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6" podStartSLOduration=1.765113558 podStartE2EDuration="3.205399802s" totalImagesPullingTime="1.440286244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:02.202329459 +0000 UTC m=+524.463329178" watchObservedRunningTime="2026-07-23 22:00:02.205399802 +0000 UTC m=+524.466399592"
Jul 23 22:00:03.188837 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:03.188794    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" event={"ID":"2bfd2089-c419-4688-8c97-5821c0128683","Type":"ContainerStarted","Data":"d0314a28fc2ac7a1618ecb1140e866d00dbdfb41daba844f47509058e71ed2e1"}
Jul 23 22:00:03.204806 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:03.204757    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" podStartSLOduration=0.646004667 podStartE2EDuration="3.204743659s" totalImagesPullingTime="2.558738992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:03.2043348 +0000 UTC m=+525.465334528" watchObservedRunningTime="2026-07-23 22:00:03.204743659 +0000 UTC m=+525.465743387"
Jul 23 22:00:04.195913 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:04.195873    2575 generic.go:356] "Generic (PLEG): container finished" podID="2bfd2089-c419-4688-8c97-5821c0128683" containerID="d0314a28fc2ac7a1618ecb1140e866d00dbdfb41daba844f47509058e71ed2e1" exitCode=7
Jul 23 22:00:04.196315 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:04.195957    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" event={"ID":"2bfd2089-c419-4688-8c97-5821c0128683","Type":"ContainerDied","Data":"d0314a28fc2ac7a1618ecb1140e866d00dbdfb41daba844f47509058e71ed2e1"}
Jul 23 22:00:04.196315 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:04.196119    2575 scope.go:117] "RemoveContainer" containerID="d0314a28fc2ac7a1618ecb1140e866d00dbdfb41daba844f47509058e71ed2e1"
Jul 23 22:00:05.203512 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:05.203406    2575 generic.go:356] "Generic (PLEG): container finished" podID="2bfd2089-c419-4688-8c97-5821c0128683" containerID="c3f8aa3dfc633a0c45ce0168798614aa7cfb9c6213f12e3d3447d724055cc47c" exitCode=7
Jul 23 22:00:05.203998 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:05.203498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" event={"ID":"2bfd2089-c419-4688-8c97-5821c0128683","Type":"ContainerDied","Data":"c3f8aa3dfc633a0c45ce0168798614aa7cfb9c6213f12e3d3447d724055cc47c"}
Jul 23 22:00:05.203998 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:05.203570    2575 scope.go:117] "RemoveContainer" containerID="d0314a28fc2ac7a1618ecb1140e866d00dbdfb41daba844f47509058e71ed2e1"
Jul 23 22:00:05.203998 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:05.203683    2575 scope.go:117] "RemoveContainer" containerID="c3f8aa3dfc633a0c45ce0168798614aa7cfb9c6213f12e3d3447d724055cc47c"
Jul 23 22:00:05.203998 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:00:05.203929    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29747400-zxl9f_odh-ai-gateway-infra(2bfd2089-c419-4688-8c97-5821c0128683)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" podUID="2bfd2089-c419-4688-8c97-5821c0128683"
Jul 23 22:00:06.211236 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:06.211206    2575 scope.go:117] "RemoveContainer" containerID="c3f8aa3dfc633a0c45ce0168798614aa7cfb9c6213f12e3d3447d724055cc47c"
Jul 23 22:00:06.211762 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:00:06.211425    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29747400-zxl9f_odh-ai-gateway-infra(2bfd2089-c419-4688-8c97-5821c0128683)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" podUID="2bfd2089-c419-4688-8c97-5821c0128683"
Jul 23 22:00:08.637421 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:08.637388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:09.088952 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:09.088863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:13.191177 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:13.191097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-rspp6"
Jul 23 22:00:21.221310 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:21.221268    2575 scope.go:117] "RemoveContainer" containerID="c3f8aa3dfc633a0c45ce0168798614aa7cfb9c6213f12e3d3447d724055cc47c"
Jul 23 22:00:22.308950 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:22.308912    2575 generic.go:356] "Generic (PLEG): container finished" podID="2bfd2089-c419-4688-8c97-5821c0128683" containerID="bb6da4bb3a85b9aaedee9742e3bfb98e1e0b46eebe8a44031ee276b5688a019f" exitCode=0
Jul 23 22:00:22.309412 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:22.308986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" event={"ID":"2bfd2089-c419-4688-8c97-5821c0128683","Type":"ContainerDied","Data":"bb6da4bb3a85b9aaedee9742e3bfb98e1e0b46eebe8a44031ee276b5688a019f"}
Jul 23 22:00:22.309412 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:22.309047    2575 scope.go:117] "RemoveContainer" containerID="c3f8aa3dfc633a0c45ce0168798614aa7cfb9c6213f12e3d3447d724055cc47c"
Jul 23 22:00:23.339413 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.339381    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"]
Jul 23 22:00:23.398128 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.398093    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"]
Jul 23 22:00:23.403083 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.403058    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.405876 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.405570    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 23 22:00:23.405876 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.405570    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 22:00:23.405876 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.405672    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 22:00:23.405876 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.405577    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4jfgf\""
Jul 23 22:00:23.412448 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.412086    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"]
Jul 23 22:00:23.428428 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.428379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:23.456179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.456152    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"
Jul 23 22:00:23.462358 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.462334    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.462458 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.462376    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.462458 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.462400    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.462536 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.462459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkzzk\" (UniqueName: \"kubernetes.io/projected/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-kube-api-access-lkzzk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.462536 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.462523    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.462611 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.462573    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.563505 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.563460    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkzzk\" (UniqueName: \"kubernetes.io/projected/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-kube-api-access-lkzzk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.563715 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.563534    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.563715 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.563575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.563715 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.563611    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.563715 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.563679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.563715 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.563702    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.564075 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.564042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.564196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.564080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.564196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.564109    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.566015 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.565997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.566184 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.566169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.570622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.570567    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkzzk\" (UniqueName: \"kubernetes.io/projected/93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea-kube-api-access-lkzzk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t\" (UID: \"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.715758 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.715717    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:23.856246 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:23.856218    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"]
Jul 23 22:00:23.858225 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:00:23.858193    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93dc1d7d_9e6f_46b5_901f_1afe5e94d0ea.slice/crio-3a05834fbd722f81d1ee305efd16480b7d2eb1ebf2ce74693214c3a212ec96ff WatchSource:0}: Error finding container 3a05834fbd722f81d1ee305efd16480b7d2eb1ebf2ce74693214c3a212ec96ff: Status 404 returned error can't find the container with id 3a05834fbd722f81d1ee305efd16480b7d2eb1ebf2ce74693214c3a212ec96ff
Jul 23 22:00:24.325566 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:24.325540    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"
Jul 23 22:00:24.325791 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:24.325585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f" event={"ID":"2bfd2089-c419-4688-8c97-5821c0128683","Type":"ContainerDied","Data":"0c5bc8892cef5994e8b29bf71a3963795a527e4c735d8862e5e5f83184d920b5"}
Jul 23 22:00:24.325791 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:24.325668    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c5bc8892cef5994e8b29bf71a3963795a527e4c735d8862e5e5f83184d920b5"
Jul 23 22:00:24.327909 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:24.327863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t" event={"ID":"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea","Type":"ContainerStarted","Data":"3a05834fbd722f81d1ee305efd16480b7d2eb1ebf2ce74693214c3a212ec96ff"}
Jul 23 22:00:24.367665 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:24.367597    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"]
Jul 23 22:00:24.376755 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:24.376723    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747400-zxl9f"]
Jul 23 22:00:26.227253 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:26.227221    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bfd2089-c419-4688-8c97-5821c0128683" path="/var/lib/kubelet/pods/2bfd2089-c419-4688-8c97-5821c0128683/volumes"
Jul 23 22:00:28.418477 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:28.418440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:37.402186 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:37.402150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t" event={"ID":"93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea","Type":"ContainerStarted","Data":"2e4c8ad0e8d7ba0fe07db60106e8408046ebb4d89e5812d9390674feee951405"}
Jul 23 22:00:37.421947 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:37.421890    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t" podStartSLOduration=1.636271837 podStartE2EDuration="14.421875002s" totalImagesPullingTime="12.785603165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:37.41910396 +0000 UTC m=+559.680103699" watchObservedRunningTime="2026-07-23 22:00:37.421875002 +0000 UTC m=+559.682874779"
Jul 23 22:00:42.776951 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.776907    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"]
Jul 23 22:00:42.777500 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.777477    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:00:42.777500 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.777501    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:00:42.777625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.777525    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:00:42.777625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.777530    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:00:42.777625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.777593    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:00:42.777625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.777605    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:00:42.779678 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.779658    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:42.782585 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.782563    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 23 22:00:42.792765 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.792736    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"]
Jul 23 22:00:42.819829 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.819787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:42.940014 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.939978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-222t4\" (UniqueName: \"kubernetes.io/projected/e81a6a8d-8e9f-47ca-90ff-000143990135-kube-api-access-222t4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:42.940178 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.940038    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:42.940178 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.940123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:42.940178 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.940152    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e81a6a8d-8e9f-47ca-90ff-000143990135-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:42.940178 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.940174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:42.940319 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:42.940238    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.041768 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.041669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-222t4\" (UniqueName: \"kubernetes.io/projected/e81a6a8d-8e9f-47ca-90ff-000143990135-kube-api-access-222t4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.041768 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.041752    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.041797    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.041816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e81a6a8d-8e9f-47ca-90ff-000143990135-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.041836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.041864    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042243 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.042222    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042305 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.042250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.042305 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.042281    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.044190 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.044164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e81a6a8d-8e9f-47ca-90ff-000143990135-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.044485 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.044464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e81a6a8d-8e9f-47ca-90ff-000143990135-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.049820 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.049797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-222t4\" (UniqueName: \"kubernetes.io/projected/e81a6a8d-8e9f-47ca-90ff-000143990135-kube-api-access-222t4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7nzlj\" (UID: \"e81a6a8d-8e9f-47ca-90ff-000143990135\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.090920 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.090873    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:43.230453 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.230421    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"]
Jul 23 22:00:43.232124 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:00:43.232095    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81a6a8d_8e9f_47ca_90ff_000143990135.slice/crio-393c0d2ea0b9038716bb29de4c60b253ff89b25433a877655623d873c4a7a4e2 WatchSource:0}: Error finding container 393c0d2ea0b9038716bb29de4c60b253ff89b25433a877655623d873c4a7a4e2: Status 404 returned error can't find the container with id 393c0d2ea0b9038716bb29de4c60b253ff89b25433a877655623d873c4a7a4e2
Jul 23 22:00:43.437380 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.437334    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj" event={"ID":"e81a6a8d-8e9f-47ca-90ff-000143990135","Type":"ContainerStarted","Data":"393c0d2ea0b9038716bb29de4c60b253ff89b25433a877655623d873c4a7a4e2"}
Jul 23 22:00:43.716803 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.716706    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:43.716803 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.716771    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:43.729579 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:43.729546    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:44.445780 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:44.445738    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj" event={"ID":"e81a6a8d-8e9f-47ca-90ff-000143990135","Type":"ContainerStarted","Data":"cb7ba44cc39bbecf0d758c94e31e7c92158a97dac7adf395c536d1c3397a331e"}
Jul 23 22:00:44.457479 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:44.457445    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t"
Jul 23 22:00:44.472263 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:44.472205    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj" podStartSLOduration=2.205012033 podStartE2EDuration="2.472190199s" totalImagesPullingTime="267.178166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:44.470320097 +0000 UTC m=+566.731319825" watchObservedRunningTime="2026-07-23 22:00:44.472190199 +0000 UTC m=+566.733189945"
Jul 23 22:00:44.602434 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:44.602397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:45.874853 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:45.874811    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:52.911833 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:52.911791    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:00:53.091132 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:53.091096    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:53.091330 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:53.091170    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:53.103778 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:53.103752    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:00:53.513683 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:00:53.513652    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7nzlj"
Jul 23 22:01:01.686685 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.686653    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"]
Jul 23 22:01:01.687417 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.687392    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:01:01.687417 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.687415    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:01:01.687603 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.687498    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bfd2089-c419-4688-8c97-5821c0128683" containerName="cleanup"
Jul 23 22:01:01.689877 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.689858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.692149 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.692130    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 22:01:01.705466 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.705442    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"]
Jul 23 22:01:01.705907 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.705883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.705997 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.705930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16d63835-b27b-4fbc-9714-ffa68e7568ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.706059 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.706005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f45g9\" (UniqueName: \"kubernetes.io/projected/16d63835-b27b-4fbc-9714-ffa68e7568ac-kube-api-access-f45g9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.706059 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.706048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.706167 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.706071    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.706167 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.706106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.722888 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.722853    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:01:01.807096 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f45g9\" (UniqueName: \"kubernetes.io/projected/16d63835-b27b-4fbc-9714-ffa68e7568ac-kube-api-access-f45g9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807277 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807277 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807277 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807277 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807277 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807268    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16d63835-b27b-4fbc-9714-ffa68e7568ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807692 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807843 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.807843 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.807728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.809404 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.809384    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16d63835-b27b-4fbc-9714-ffa68e7568ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.809757 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.809741    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16d63835-b27b-4fbc-9714-ffa68e7568ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.814309 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.814289    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f45g9\" (UniqueName: \"kubernetes.io/projected/16d63835-b27b-4fbc-9714-ffa68e7568ac-kube-api-access-f45g9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8\" (UID: \"16d63835-b27b-4fbc-9714-ffa68e7568ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:01.999523 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:01.999423    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:02.140141 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:02.140110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"]
Jul 23 22:01:02.141571 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:01:02.141542    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16d63835_b27b_4fbc_9714_ffa68e7568ac.slice/crio-0bbab8e73e417fa88dbff0887f944a44c2938dbe119e0fbff83896c019307bfb WatchSource:0}: Error finding container 0bbab8e73e417fa88dbff0887f944a44c2938dbe119e0fbff83896c019307bfb: Status 404 returned error can't find the container with id 0bbab8e73e417fa88dbff0887f944a44c2938dbe119e0fbff83896c019307bfb
Jul 23 22:01:02.553111 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:02.553072    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8" event={"ID":"16d63835-b27b-4fbc-9714-ffa68e7568ac","Type":"ContainerStarted","Data":"0bbab8e73e417fa88dbff0887f944a44c2938dbe119e0fbff83896c019307bfb"}
Jul 23 22:01:03.560368 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:03.560328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8" event={"ID":"16d63835-b27b-4fbc-9714-ffa68e7568ac","Type":"ContainerStarted","Data":"b87ab9a8eb28ada4e434cd64229beb3574452264ce5d01b07a21ab977a9c5120"}
Jul 23 22:01:03.580440 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:03.580385    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8" podStartSLOduration=2.2095346 podStartE2EDuration="2.580370019s" totalImagesPullingTime="370.835419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:01:03.577904695 +0000 UTC m=+585.838904465" watchObservedRunningTime="2026-07-23 22:01:03.580370019 +0000 UTC m=+585.841369746"
Jul 23 22:01:05.115582 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:05.115539    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:01:05.216363 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:05.216322    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:01:09.668402 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:09.668355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:01:09.768698 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:09.768659    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:01:12.000314 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:12.000275    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:12.000771 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:12.000540    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:12.012842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:12.012818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:12.625299 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:12.625269    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8"
Jul 23 22:01:18.239693 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:18.239665    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:01:18.240938 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:18.240916    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:01:47.300509 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:01:47.300422    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:02:05.107184 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.107143    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58c8d7c7bb-p6djq"]
Jul 23 22:02:05.107668 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.107341    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq" podUID="0b59fba4-932a-46cb-8aff-7e79481f4bfa" containerName="authorino" containerID="cri-o://66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d" gracePeriod=30
Jul 23 22:02:05.372557 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.372532    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 22:02:05.490566 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.490537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt5qh\" (UniqueName: \"kubernetes.io/projected/0b59fba4-932a-46cb-8aff-7e79481f4bfa-kube-api-access-xt5qh\") pod \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") "
Jul 23 22:02:05.490743 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.490583    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b59fba4-932a-46cb-8aff-7e79481f4bfa-tls-cert\") pod \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\" (UID: \"0b59fba4-932a-46cb-8aff-7e79481f4bfa\") "
Jul 23 22:02:05.492792 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.492758    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b59fba4-932a-46cb-8aff-7e79481f4bfa-kube-api-access-xt5qh" (OuterVolumeSpecName: "kube-api-access-xt5qh") pod "0b59fba4-932a-46cb-8aff-7e79481f4bfa" (UID: "0b59fba4-932a-46cb-8aff-7e79481f4bfa"). InnerVolumeSpecName "kube-api-access-xt5qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:02:05.501921 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.501897    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b59fba4-932a-46cb-8aff-7e79481f4bfa-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0b59fba4-932a-46cb-8aff-7e79481f4bfa" (UID: "0b59fba4-932a-46cb-8aff-7e79481f4bfa"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:02:05.592184 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.592149    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt5qh\" (UniqueName: \"kubernetes.io/projected/0b59fba4-932a-46cb-8aff-7e79481f4bfa-kube-api-access-xt5qh\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:02:05.592184 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.592178    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b59fba4-932a-46cb-8aff-7e79481f4bfa-tls-cert\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:02:05.618869 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.618788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:02:05.722239 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.722200    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:02:05.927468 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.927376    2575 generic.go:356] "Generic (PLEG): container finished" podID="0b59fba4-932a-46cb-8aff-7e79481f4bfa" containerID="66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d" exitCode=0
Jul 23 22:02:05.927468 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.927421    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq"
Jul 23 22:02:05.927468 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.927456    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq" event={"ID":"0b59fba4-932a-46cb-8aff-7e79481f4bfa","Type":"ContainerDied","Data":"66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d"}
Jul 23 22:02:05.927770 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.927512    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c8d7c7bb-p6djq" event={"ID":"0b59fba4-932a-46cb-8aff-7e79481f4bfa","Type":"ContainerDied","Data":"44f73cb7f8c39c9b27a8b5c368c71ca5cfc680e78feb43f3107ea5e4c4f96899"}
Jul 23 22:02:05.927770 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.927534    2575 scope.go:117] "RemoveContainer" containerID="66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d"
Jul 23 22:02:05.937976 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.937957    2575 scope.go:117] "RemoveContainer" containerID="66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d"
Jul 23 22:02:05.938245 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:02:05.938228    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d\": container with ID starting with 66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d not found: ID does not exist" containerID="66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d"
Jul 23 22:02:05.938286 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.938254    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d"} err="failed to get container status \"66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d\": rpc error: code = NotFound desc = could not find container \"66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d\": container with ID starting with 66855fd20c0630745b512260c8af3969d5bd731e29fcf5941d37c5519b11d10d not found: ID does not exist"
Jul 23 22:02:05.949724 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.949694    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58c8d7c7bb-p6djq"]
Jul 23 22:02:05.956101 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:05.956076    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58c8d7c7bb-p6djq"]
Jul 23 22:02:06.227720 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:06.227626    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b59fba4-932a-46cb-8aff-7e79481f4bfa" path="/var/lib/kubelet/pods/0b59fba4-932a-46cb-8aff-7e79481f4bfa/volumes"
Jul 23 22:02:06.843934 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:06.843898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:02:06.952569 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:06.952524    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:02:51.894383 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:51.894344    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:02:51.991418 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:02:51.991384    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:03:18.432941 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:03:18.432740    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:12.085518 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:12.085473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:13.418842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:13.418808    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:31.556421 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:31.556384    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:43.896856 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:43.896772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:44.003955 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:44.003915    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:44.117283 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:44.117245    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:49.910131 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:49.910091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:49.972045 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:49.971999    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:50.067572 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:50.067541    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:50.158431 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:50.158400    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:50.306909 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:50.306817    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:56.026984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:56.026945    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:04:56.129418 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:04:56.129389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:06:18.277522 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:06:18.277440    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:06:18.279590 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:06:18.279565    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:07:18.621330 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:07:18.621296    2575 scope.go:117] "RemoveContainer" containerID="bb6da4bb3a85b9aaedee9742e3bfb98e1e0b46eebe8a44031ee276b5688a019f"
Jul 23 22:08:03.432503 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:03.432461    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:08:29.823052 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:29.823003    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:08:43.624355 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:43.624315    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:08:43.726173 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:43.726130    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:08:51.380932 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:51.380893    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"]
Jul 23 22:08:51.381432 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:51.381106    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j" podUID="956713b4-2cd5-409e-bc9d-d28972e4d429" containerName="manager" containerID="cri-o://d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee" gracePeriod=10
Jul 23 22:08:54.547493 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.547468    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 22:08:54.650656 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.650594    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6xwr\" (UniqueName: \"kubernetes.io/projected/956713b4-2cd5-409e-bc9d-d28972e4d429-kube-api-access-b6xwr\") pod \"956713b4-2cd5-409e-bc9d-d28972e4d429\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") "
Jul 23 22:08:54.650830 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.650678    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/956713b4-2cd5-409e-bc9d-d28972e4d429-extensions-socket-volume\") pod \"956713b4-2cd5-409e-bc9d-d28972e4d429\" (UID: \"956713b4-2cd5-409e-bc9d-d28972e4d429\") "
Jul 23 22:08:54.651117 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.651089    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/956713b4-2cd5-409e-bc9d-d28972e4d429-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "956713b4-2cd5-409e-bc9d-d28972e4d429" (UID: "956713b4-2cd5-409e-bc9d-d28972e4d429"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:08:54.652786 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.652760    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/956713b4-2cd5-409e-bc9d-d28972e4d429-kube-api-access-b6xwr" (OuterVolumeSpecName: "kube-api-access-b6xwr") pod "956713b4-2cd5-409e-bc9d-d28972e4d429" (UID: "956713b4-2cd5-409e-bc9d-d28972e4d429"). InnerVolumeSpecName "kube-api-access-b6xwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:08:54.751795 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.751753    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6xwr\" (UniqueName: \"kubernetes.io/projected/956713b4-2cd5-409e-bc9d-d28972e4d429-kube-api-access-b6xwr\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:08:54.751795 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:54.751788    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/956713b4-2cd5-409e-bc9d-d28972e4d429-extensions-socket-volume\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:08:55.265317 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.265281    2575 generic.go:356] "Generic (PLEG): container finished" podID="956713b4-2cd5-409e-bc9d-d28972e4d429" containerID="d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee" exitCode=0
Jul 23 22:08:55.265495 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.265338    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"
Jul 23 22:08:55.265495 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.265365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j" event={"ID":"956713b4-2cd5-409e-bc9d-d28972e4d429","Type":"ContainerDied","Data":"d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee"}
Jul 23 22:08:55.265495 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.265417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j" event={"ID":"956713b4-2cd5-409e-bc9d-d28972e4d429","Type":"ContainerDied","Data":"255f52eed2c0c9ff784897068745ae0dac8cfe0682f3d70fc91fe7197bf5f06d"}
Jul 23 22:08:55.265495 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.265438    2575 scope.go:117] "RemoveContainer" containerID="d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee"
Jul 23 22:08:55.278584 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.278539    2575 scope.go:117] "RemoveContainer" containerID="d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee"
Jul 23 22:08:55.279014 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:08:55.278992    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee\": container with ID starting with d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee not found: ID does not exist" containerID="d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee"
Jul 23 22:08:55.279134 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.279019    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee"} err="failed to get container status \"d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee\": rpc error: code = NotFound desc = could not find container \"d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee\": container with ID starting with d699098ca77bc0b28fe1ca88a96f3ab3980d41d0b6244067f97982c977986bee not found: ID does not exist"
Jul 23 22:08:55.287908 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.287882    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"]
Jul 23 22:08:55.293422 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:55.293399    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l947j"]
Jul 23 22:08:56.227159 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:08:56.227127    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="956713b4-2cd5-409e-bc9d-d28972e4d429" path="/var/lib/kubelet/pods/956713b4-2cd5-409e-bc9d-d28972e4d429/volumes"
Jul 23 22:09:52.600528 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.600494    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"]
Jul 23 22:09:52.601012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.600914    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b59fba4-932a-46cb-8aff-7e79481f4bfa" containerName="authorino"
Jul 23 22:09:52.601012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.600926    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b59fba4-932a-46cb-8aff-7e79481f4bfa" containerName="authorino"
Jul 23 22:09:52.601012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.600940    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="956713b4-2cd5-409e-bc9d-d28972e4d429" containerName="manager"
Jul 23 22:09:52.601012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.600946    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="956713b4-2cd5-409e-bc9d-d28972e4d429" containerName="manager"
Jul 23 22:09:52.601012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.601005    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b59fba4-932a-46cb-8aff-7e79481f4bfa" containerName="authorino"
Jul 23 22:09:52.601012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.601016    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="956713b4-2cd5-409e-bc9d-d28972e4d429" containerName="manager"
Jul 23 22:09:52.603140 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.603123    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.605329 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.605309    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r28lz\""
Jul 23 22:09:52.616248 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.616218    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"]
Jul 23 22:09:52.669255 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.669221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wplkg\" (UID: \"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.669439 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.669260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjhfm\" (UniqueName: \"kubernetes.io/projected/9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724-kube-api-access-kjhfm\") pod \"kuadrant-operator-controller-manager-86878c769f-wplkg\" (UID: \"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.770091 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.770046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wplkg\" (UID: \"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.770091 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.770089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjhfm\" (UniqueName: \"kubernetes.io/projected/9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724-kube-api-access-kjhfm\") pod \"kuadrant-operator-controller-manager-86878c769f-wplkg\" (UID: \"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.770447 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.770426    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wplkg\" (UID: \"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.779175 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.779145    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjhfm\" (UniqueName: \"kubernetes.io/projected/9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724-kube-api-access-kjhfm\") pod \"kuadrant-operator-controller-manager-86878c769f-wplkg\" (UID: \"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:52.913512 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:52.913422    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:53.057134 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:53.057105    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"]
Jul 23 22:09:53.060168 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:09:53.060140    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0ba7bc_4ee0_403f_b2d9_6e92dd20d724.slice/crio-83f2efe3ddf9042753941e0ac8d5908a588fc26c18503715c9092a12e39071ec WatchSource:0}: Error finding container 83f2efe3ddf9042753941e0ac8d5908a588fc26c18503715c9092a12e39071ec: Status 404 returned error can't find the container with id 83f2efe3ddf9042753941e0ac8d5908a588fc26c18503715c9092a12e39071ec
Jul 23 22:09:53.591244 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:53.591205    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg" event={"ID":"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724","Type":"ContainerStarted","Data":"dd531ea7ff9f134df908b2fccd6a0d1192c3816d1f055f7217b11552b7ed2608"}
Jul 23 22:09:53.591421 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:53.591262    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg" event={"ID":"9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724","Type":"ContainerStarted","Data":"83f2efe3ddf9042753941e0ac8d5908a588fc26c18503715c9092a12e39071ec"}
Jul 23 22:09:53.591421 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:53.591291    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:09:53.612394 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:09:53.612340    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg" podStartSLOduration=1.612324541 podStartE2EDuration="1.612324541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:09:53.611163284 +0000 UTC m=+1115.872163038" watchObservedRunningTime="2026-07-23 22:09:53.612324541 +0000 UTC m=+1115.873324268"
Jul 23 22:10:04.599973 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:10:04.599941    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wplkg"
Jul 23 22:10:26.097572 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:10:26.097537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:11:18.314081 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:11:18.314052    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:11:18.318338 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:11:18.318317    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:12:49.864540 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:49.864498    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:12:49.925957 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:49.925920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:12:59.397219 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.397184    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"]
Jul 23 22:12:59.400943 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.400922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.403276 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.403258    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-bc6d27b0-kserve-self-signed-certs\""
Jul 23 22:12:59.419963 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.419931    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"]
Jul 23 22:12:59.425889 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.425857    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:12:59.552290 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.552249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-model-cache\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.552485 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.552309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-dshm\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.552485 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.552392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhqtx\" (UniqueName: \"kubernetes.io/projected/625843a5-bc27-4143-8e17-d798d42e904f-kube-api-access-fhqtx\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.552485 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.552433    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-tmp-dir\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.552676 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.552559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-home\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.552676 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.552598    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625843a5-bc27-4143-8e17-d798d42e904f-tls-certs\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.653543 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-home\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.653543 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625843a5-bc27-4143-8e17-d798d42e904f-tls-certs\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.653819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-model-cache\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.653819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-dshm\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.653819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653640    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhqtx\" (UniqueName: \"kubernetes.io/projected/625843a5-bc27-4143-8e17-d798d42e904f-kube-api-access-fhqtx\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.653819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-tmp-dir\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.654026 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.653977    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-home\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.654086 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.654016    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-model-cache\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.654191 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.654172    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-tmp-dir\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.656065 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.656039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-dshm\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.656363 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.656346    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625843a5-bc27-4143-8e17-d798d42e904f-tls-certs\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.661728 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.661702    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhqtx\" (UniqueName: \"kubernetes.io/projected/625843a5-bc27-4143-8e17-d798d42e904f-kube-api-access-fhqtx\") pod \"e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") " pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.711675 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.711618    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:12:59.856747 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:12:59.856717    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"]
Jul 23 22:12:59.858316 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:12:59.858282    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod625843a5_bc27_4143_8e17_d798d42e904f.slice/crio-7c75e605ab61b5687fab230d7cb20e767e688cc53412fef45235bd8c6e2cb16c WatchSource:0}: Error finding container 7c75e605ab61b5687fab230d7cb20e767e688cc53412fef45235bd8c6e2cb16c: Status 404 returned error can't find the container with id 7c75e605ab61b5687fab230d7cb20e767e688cc53412fef45235bd8c6e2cb16c
Jul 23 22:13:00.662147 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:00.662107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879" event={"ID":"625843a5-bc27-4143-8e17-d798d42e904f","Type":"ContainerStarted","Data":"da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9"}
Jul 23 22:13:00.662528 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:00.662157    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879" event={"ID":"625843a5-bc27-4143-8e17-d798d42e904f","Type":"ContainerStarted","Data":"7c75e605ab61b5687fab230d7cb20e767e688cc53412fef45235bd8c6e2cb16c"}
Jul 23 22:13:00.680522 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:00.680462    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879" podStartSLOduration=1.680446045 podStartE2EDuration="1.680446045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:13:00.678610556 +0000 UTC m=+1302.939610283" watchObservedRunningTime="2026-07-23 22:13:00.680446045 +0000 UTC m=+1302.941445789"
Jul 23 22:13:03.796601 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:03.796558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:13:09.712642 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:09.712597    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:13:09.713149 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:09.712659    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:13:09.726704 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:09.726675    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:13:09.742104 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:09.742072    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:13:33.644384 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:33.644347    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"]
Jul 23 22:13:33.667699 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:33.667669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:13:34.798403 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:34.798371    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:13:48.666465 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.666368    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879" podUID="625843a5-bc27-4143-8e17-d798d42e904f" containerName="main" containerID="cri-o://da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9" gracePeriod=45
Jul 23 22:13:48.929049 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.928983    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:13:48.948425 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.948390    2575 generic.go:356] "Generic (PLEG): container finished" podID="625843a5-bc27-4143-8e17-d798d42e904f" containerID="da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9" exitCode=0
Jul 23 22:13:48.948592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.948437    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"
Jul 23 22:13:48.948592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.948487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879" event={"ID":"625843a5-bc27-4143-8e17-d798d42e904f","Type":"ContainerDied","Data":"da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9"}
Jul 23 22:13:48.948592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.948532    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879" event={"ID":"625843a5-bc27-4143-8e17-d798d42e904f","Type":"ContainerDied","Data":"7c75e605ab61b5687fab230d7cb20e767e688cc53412fef45235bd8c6e2cb16c"}
Jul 23 22:13:48.948592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.948558    2575 scope.go:117] "RemoveContainer" containerID="da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9"
Jul 23 22:13:48.958622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.958599    2575 scope.go:117] "RemoveContainer" containerID="da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9"
Jul 23 22:13:48.958943 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:13:48.958919    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9\": container with ID starting with da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9 not found: ID does not exist" containerID="da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9"
Jul 23 22:13:48.959008 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:48.958954    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9"} err="failed to get container status \"da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9\": rpc error: code = NotFound desc = could not find container \"da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9\": container with ID starting with da5fe3a19d3e416954a536cd6d544a9c0b904e4698ecde87243f25d70714e2b9 not found: ID does not exist"
Jul 23 22:13:49.016896 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.016866    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-tmp-dir\") pod \"625843a5-bc27-4143-8e17-d798d42e904f\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") "
Jul 23 22:13:49.017015 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.016907    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-home\") pod \"625843a5-bc27-4143-8e17-d798d42e904f\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") "
Jul 23 22:13:49.017015 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.016999    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-model-cache\") pod \"625843a5-bc27-4143-8e17-d798d42e904f\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") "
Jul 23 22:13:49.017089 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017018    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-dshm\") pod \"625843a5-bc27-4143-8e17-d798d42e904f\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") "
Jul 23 22:13:49.017089 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017043    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhqtx\" (UniqueName: \"kubernetes.io/projected/625843a5-bc27-4143-8e17-d798d42e904f-kube-api-access-fhqtx\") pod \"625843a5-bc27-4143-8e17-d798d42e904f\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") "
Jul 23 22:13:49.017089 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017065    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625843a5-bc27-4143-8e17-d798d42e904f-tls-certs\") pod \"625843a5-bc27-4143-8e17-d798d42e904f\" (UID: \"625843a5-bc27-4143-8e17-d798d42e904f\") "
Jul 23 22:13:49.017219 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017140    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "625843a5-bc27-4143-8e17-d798d42e904f" (UID: "625843a5-bc27-4143-8e17-d798d42e904f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:13:49.017219 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017188    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-home" (OuterVolumeSpecName: "home") pod "625843a5-bc27-4143-8e17-d798d42e904f" (UID: "625843a5-bc27-4143-8e17-d798d42e904f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:13:49.017326 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-model-cache" (OuterVolumeSpecName: "model-cache") pod "625843a5-bc27-4143-8e17-d798d42e904f" (UID: "625843a5-bc27-4143-8e17-d798d42e904f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:13:49.017386 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017354    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-tmp-dir\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:13:49.017386 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.017368    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-home\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:13:49.019288 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.019263    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625843a5-bc27-4143-8e17-d798d42e904f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "625843a5-bc27-4143-8e17-d798d42e904f" (UID: "625843a5-bc27-4143-8e17-d798d42e904f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:13:49.019288 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.019283    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625843a5-bc27-4143-8e17-d798d42e904f-kube-api-access-fhqtx" (OuterVolumeSpecName: "kube-api-access-fhqtx") pod "625843a5-bc27-4143-8e17-d798d42e904f" (UID: "625843a5-bc27-4143-8e17-d798d42e904f"). InnerVolumeSpecName "kube-api-access-fhqtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:13:49.019422 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.019339    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-dshm" (OuterVolumeSpecName: "dshm") pod "625843a5-bc27-4143-8e17-d798d42e904f" (UID: "625843a5-bc27-4143-8e17-d798d42e904f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:13:49.118105 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.118053    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-model-cache\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:13:49.118105 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.118100    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625843a5-bc27-4143-8e17-d798d42e904f-dshm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:13:49.118105 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.118110    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhqtx\" (UniqueName: \"kubernetes.io/projected/625843a5-bc27-4143-8e17-d798d42e904f-kube-api-access-fhqtx\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:13:49.118105 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.118122    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625843a5-bc27-4143-8e17-d798d42e904f-tls-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:13:49.272515 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.272487    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"]
Jul 23 22:13:49.274780 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:49.274759    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-bc6d27b0-kserve-65b987f7cf-d2879"]
Jul 23 22:13:50.227960 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:13:50.227919    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625843a5-bc27-4143-8e17-d798d42e904f" path="/var/lib/kubelet/pods/625843a5-bc27-4143-8e17-d798d42e904f/volumes"
Jul 23 22:15:55.304174 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:15:55.304145    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:16:03.988545 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:16:03.988510    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:16:18.353704 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:16:18.353672    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:16:18.357147 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:16:18.357123    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:16:49.520690 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:16:49.520577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:16:57.112454 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:16:57.112411    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:16:57.221256 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:16:57.221224    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:12.482212 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:12.482164    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:15.769062 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:15.769028    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:15.875285 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:15.875247    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:16.258547 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:16.258513    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:22.901916 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:22.901886    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:22.968993 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:22.968957    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:23.191448 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:23.191364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:23.570405 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:23.570318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:23.686358 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:23.686327    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:26.426412 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:26.426365    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:26.541428 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:26.541392    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:26.640146 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:26.640115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:26.955528 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:26.955486    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:27.290764 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:27.290685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:27.405983 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:27.405951    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:32.545963 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:32.545918    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:17:32.617786 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:17:32.617750    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:02.598426 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:02.598389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:07.128366 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:07.128328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:07.295963 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:07.295931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:07.601981 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:07.601949    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:07.850061 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:07.850024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:07.951089 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:07.951008    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:08.062022 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:08.061984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:08.407445 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:08.407400    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:08.728256 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:08.728162    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:10.711422 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:10.711346    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:10.824829 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:10.824796    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:10.925900 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:10.925842    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:11.357026 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:11.356980    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:11.457585 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:11.457540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:46.248917 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:46.248880    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:50.854919 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:50.854881    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:51.032517 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:51.032475    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:51.252165 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:51.252075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:51.518245 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:51.518157    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:51.848280 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:51.848247    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:51.985687 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:51.985654    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:53.828389 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:53.828337    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:53.932884 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:53.932846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:54.028371 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:54.028331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:54.248961 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:54.248924    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:54.655946 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:54.655916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:18:54.756669 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:18:54.756616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:19.712352 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.712312    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"]
Jul 23 22:19:19.712777 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.712761    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625843a5-bc27-4143-8e17-d798d42e904f" containerName="main"
Jul 23 22:19:19.712777 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.712778    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="625843a5-bc27-4143-8e17-d798d42e904f" containerName="main"
Jul 23 22:19:19.712855 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.712844    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="625843a5-bc27-4143-8e17-d798d42e904f" containerName="main"
Jul 23 22:19:19.715975 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.715951    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:19.718418 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.718390    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:19:19.724844 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.724423    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"]
Jul 23 22:19:19.855908 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.855868    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5776abec-f19f-44bb-bbdc-225120688a6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c965c007-t6v9c\" (UID: \"5776abec-f19f-44bb-bbdc-225120688a6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:19.956702 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.956623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5776abec-f19f-44bb-bbdc-225120688a6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c965c007-t6v9c\" (UID: \"5776abec-f19f-44bb-bbdc-225120688a6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:19.957263 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:19.957242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5776abec-f19f-44bb-bbdc-225120688a6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c965c007-t6v9c\" (UID: \"5776abec-f19f-44bb-bbdc-225120688a6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:20.030154 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:20.030059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:20.169133 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:20.169104    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"]
Jul 23 22:19:20.170642 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:19:20.170596    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5776abec_f19f_44bb_bbdc_225120688a6c.slice/crio-81c2ed1255fdd2b636df1add0dfa2264a47491b447a8878839e98f3f54252da9 WatchSource:0}: Error finding container 81c2ed1255fdd2b636df1add0dfa2264a47491b447a8878839e98f3f54252da9: Status 404 returned error can't find the container with id 81c2ed1255fdd2b636df1add0dfa2264a47491b447a8878839e98f3f54252da9
Jul 23 22:19:20.842861 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:20.842828    2575 generic.go:356] "Generic (PLEG): container finished" podID="5776abec-f19f-44bb-bbdc-225120688a6c" containerID="f3dfd88019a4427e399e218d718acc55f7f2450e19d6110ac504680766901875" exitCode=0
Jul 23 22:19:20.843172 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:20.842883    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c" event={"ID":"5776abec-f19f-44bb-bbdc-225120688a6c","Type":"ContainerDied","Data":"f3dfd88019a4427e399e218d718acc55f7f2450e19d6110ac504680766901875"}
Jul 23 22:19:20.843172 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:20.842938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c" event={"ID":"5776abec-f19f-44bb-bbdc-225120688a6c","Type":"ContainerStarted","Data":"81c2ed1255fdd2b636df1add0dfa2264a47491b447a8878839e98f3f54252da9"}
Jul 23 22:19:21.994056 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:21.994032    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:22.077420 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:22.077383    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5776abec-f19f-44bb-bbdc-225120688a6c-maas-api-service-ca\") pod \"5776abec-f19f-44bb-bbdc-225120688a6c\" (UID: \"5776abec-f19f-44bb-bbdc-225120688a6c\") "
Jul 23 22:19:22.077784 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:22.077760    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5776abec-f19f-44bb-bbdc-225120688a6c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5776abec-f19f-44bb-bbdc-225120688a6c" (UID: "5776abec-f19f-44bb-bbdc-225120688a6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:19:22.178941 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:22.178850    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5776abec-f19f-44bb-bbdc-225120688a6c-maas-api-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:19:22.855710 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:22.855682    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"
Jul 23 22:19:22.856007 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:22.855682    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c" event={"ID":"5776abec-f19f-44bb-bbdc-225120688a6c","Type":"ContainerDied","Data":"81c2ed1255fdd2b636df1add0dfa2264a47491b447a8878839e98f3f54252da9"}
Jul 23 22:19:22.856007 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:22.855818    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81c2ed1255fdd2b636df1add0dfa2264a47491b447a8878839e98f3f54252da9"
Jul 23 22:19:30.278910 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:30.278874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:34.786920 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:34.786883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:34.796598 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:34.796567    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"]
Jul 23 22:19:34.799914 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:34.799887    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c965c007-t6v9c"]
Jul 23 22:19:35.747839 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:35.747801    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:35.849438 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:35.849398    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:36.032626 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:36.032547    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:36.227580 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:36.227547    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5776abec-f19f-44bb-bbdc-225120688a6c" path="/var/lib/kubelet/pods/5776abec-f19f-44bb-bbdc-225120688a6c/volumes"
Jul 23 22:19:36.291956 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:36.291868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:36.536880 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:36.536828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:36.875343 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:36.875299    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:37.004223 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:37.004181    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:39.148524 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:39.148485    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:39.299364 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:39.299329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:39.400661 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:39.400548    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:39.780047 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:39.779964    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:39.881236 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:39.881197    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:19:39.978897 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:19:39.978863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:14.642040 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:14.642002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:19.050102 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:19.050065    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:19.211618 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:19.211585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:19.520493 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:19.520458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:19.965714 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:19.965683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:20.061667 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:20.061615    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:20.160786 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:20.160749    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:21.666553 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:21.666515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:27.472897 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:27.472845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:27.591485 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:27.591451    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:27.737286 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:27.737192    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:27.832757 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:27.832726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:28.221964 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:28.221929    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:20:54.246542 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:20:54.246508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:00.646374 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:00.646331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:00.751135 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:00.751102    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:00.840394 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:00.840349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:16.810150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:16.810065    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:18.390234 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:18.390207    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:21:18.395134 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:18.395109    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:21:22.983867 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:22.983826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:23.094076 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:23.094036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:23.187432 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:23.187399    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:33.754819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:33.754731    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:34.606923 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:34.606886    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:40.898994 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:40.898959    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:41.008411 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:41.008381    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:41.106309 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:41.106279    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:41.198286 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:41.198196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:42.069533 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:42.069501    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:42.163391 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:42.163359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:42.266924 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:42.266881    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:43.981910 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:43.981871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:49.995683 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:49.995625    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:50.059189 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:50.059157    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:50.706699 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:50.706624    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:56.723987 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:56.723950    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:56.824761 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:56.824726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:56.983792 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:56.983713    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:57.335479 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:57.335395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:21:57.514250 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:21:57.514219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:00.516596 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:00.516562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:00.605601 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:00.605571    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:00.681333 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:00.681295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:01.013843 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:01.013812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:01.376029 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:01.375993    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:01.478694 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:01.478662    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:04.928179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:04.928141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:05.617976 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:05.617942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:05.741026 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:05.740987    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:06.164069 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:06.164036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:12.071471 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:12.071434    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:12.147615 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:12.147583    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:22.730679 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:22.730640    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:27.533007 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:27.532968    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:27.935910 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:27.935874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:33.919723 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:33.919684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:33.991422 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:33.991388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:34.221778 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:34.221676    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:34.450932 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:34.450899    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:34.577793 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:34.577682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:34.675999 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:34.675967    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:34.810276 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:34.810241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:40.425553 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:40.425452    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:40.528388 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:40.528352    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:40.631740 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:40.631709    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:40.799096 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:40.799003    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:46.607452 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:46.607389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:46.679692 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:46.679656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:53.557467 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:53.557433    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:53.880768 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:53.880730    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:54.017819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:54.017781    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:56.722989 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:56.722959    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:56.810572 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:56.810537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:57.053528 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:57.053441    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:57.175114 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:57.175078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:57.826287 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:57.826259    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:22:57.928822 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:22:57.928788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:01.211296 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:01.211262    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:16.095553 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.095507    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"]
Jul 23 22:23:16.096120 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.096099    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5776abec-f19f-44bb-bbdc-225120688a6c" containerName="revoke-keys"
Jul 23 22:23:16.096166 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.096121    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5776abec-f19f-44bb-bbdc-225120688a6c" containerName="revoke-keys"
Jul 23 22:23:16.096205 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.096187    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5776abec-f19f-44bb-bbdc-225120688a6c" containerName="revoke-keys"
Jul 23 22:23:16.099351 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.099335    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.104284 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.104263    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0578d301\"/\"kube-root-ca.crt\""
Jul 23 22:23:16.105097 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.105076    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0578d301\"/\"default-dockercfg-jp5jr\""
Jul 23 22:23:16.105202 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.105078    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0578d301\"/\"openshift-service-ca.crt\""
Jul 23 22:23:16.105202 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.105148    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0578d301\"/\"e2e-lifecycle-model-0578d301-kserve-self-signed-certs\""
Jul 23 22:23:16.116716 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.116685    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"]
Jul 23 22:23:16.129102 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.129067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:16.176694 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.176653    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8cc68de-6a98-4803-a8da-fe38424e897e-tls-certs\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.176694 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.176694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-home\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.176953 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.176780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-dshm\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.176953 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.176870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tw4h\" (UniqueName: \"kubernetes.io/projected/d8cc68de-6a98-4803-a8da-fe38424e897e-kube-api-access-4tw4h\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.176953 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.176903    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-model-cache\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.176953 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.176930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-tmp-dir\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277388 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-model-cache\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277388 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-tmp-dir\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277654 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277424    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8cc68de-6a98-4803-a8da-fe38424e897e-tls-certs\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277654 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-home\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277654 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-dshm\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277850 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277627    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tw4h\" (UniqueName: \"kubernetes.io/projected/d8cc68de-6a98-4803-a8da-fe38424e897e-kube-api-access-4tw4h\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277850 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277834    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-model-cache\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.277999 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.277978    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-tmp-dir\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.278184 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.278163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-home\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.279890 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.279864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-dshm\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.279999 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.279970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8cc68de-6a98-4803-a8da-fe38424e897e-tls-certs\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.285973 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.285951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tw4h\" (UniqueName: \"kubernetes.io/projected/d8cc68de-6a98-4803-a8da-fe38424e897e-kube-api-access-4tw4h\") pod \"e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") " pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.409293 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.409209    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:16.558790 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:16.558666    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"]
Jul 23 22:23:16.561803 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:23:16.561754    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8cc68de_6a98_4803_a8da_fe38424e897e.slice/crio-0f33f713ec6c555ae14e4f345736e160b127bbde923684471f8f50cfe75b55da WatchSource:0}: Error finding container 0f33f713ec6c555ae14e4f345736e160b127bbde923684471f8f50cfe75b55da: Status 404 returned error can't find the container with id 0f33f713ec6c555ae14e4f345736e160b127bbde923684471f8f50cfe75b55da
Jul 23 22:23:17.210508 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:17.210466    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f" event={"ID":"d8cc68de-6a98-4803-a8da-fe38424e897e","Type":"ContainerStarted","Data":"9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be"}
Jul 23 22:23:17.210948 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:17.210516    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f" event={"ID":"d8cc68de-6a98-4803-a8da-fe38424e897e","Type":"ContainerStarted","Data":"0f33f713ec6c555ae14e4f345736e160b127bbde923684471f8f50cfe75b55da"}
Jul 23 22:23:17.232427 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:17.232358    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f" podStartSLOduration=1.2323303110000001 podStartE2EDuration="1.232330311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:23:17.231735564 +0000 UTC m=+1919.492735292" watchObservedRunningTime="2026-07-23 22:23:17.232330311 +0000 UTC m=+1919.493330048"
Jul 23 22:23:18.959798 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:18.959752    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:19.060536 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:19.060504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:23:26.410017 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:26.409976    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:26.410388 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:26.410038    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:26.423075 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:26.423048    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:23:27.279791 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:23:27.279753    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:24:00.583724 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:00.583681    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:05.178570 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:05.178535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:07.564547 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:07.564507    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:12.583417 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:12.583333    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"]
Jul 23 22:24:12.691392 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:12.691356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:27.604162 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:27.604109    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f" podUID="d8cc68de-6a98-4803-a8da-fe38424e897e" containerName="main" containerID="cri-o://9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be" gracePeriod=45
Jul 23 22:24:27.865601 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:27.865538    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:24:28.027515 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027478    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-dshm\") pod \"d8cc68de-6a98-4803-a8da-fe38424e897e\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") "
Jul 23 22:24:28.027747 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027532    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8cc68de-6a98-4803-a8da-fe38424e897e-tls-certs\") pod \"d8cc68de-6a98-4803-a8da-fe38424e897e\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") "
Jul 23 22:24:28.027747 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027584    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-model-cache\") pod \"d8cc68de-6a98-4803-a8da-fe38424e897e\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") "
Jul 23 22:24:28.027747 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027609    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-tmp-dir\") pod \"d8cc68de-6a98-4803-a8da-fe38424e897e\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") "
Jul 23 22:24:28.027924 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027765    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tw4h\" (UniqueName: \"kubernetes.io/projected/d8cc68de-6a98-4803-a8da-fe38424e897e-kube-api-access-4tw4h\") pod \"d8cc68de-6a98-4803-a8da-fe38424e897e\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") "
Jul 23 22:24:28.027924 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027818    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-home\") pod \"d8cc68de-6a98-4803-a8da-fe38424e897e\" (UID: \"d8cc68de-6a98-4803-a8da-fe38424e897e\") "
Jul 23 22:24:28.027924 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027886    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d8cc68de-6a98-4803-a8da-fe38424e897e" (UID: "d8cc68de-6a98-4803-a8da-fe38424e897e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:24:28.027924 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.027899    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-model-cache" (OuterVolumeSpecName: "model-cache") pod "d8cc68de-6a98-4803-a8da-fe38424e897e" (UID: "d8cc68de-6a98-4803-a8da-fe38424e897e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:24:28.028142 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.028116    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-home" (OuterVolumeSpecName: "home") pod "d8cc68de-6a98-4803-a8da-fe38424e897e" (UID: "d8cc68de-6a98-4803-a8da-fe38424e897e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:24:28.028217 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.028129    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-model-cache\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:24:28.028217 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.028161    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-tmp-dir\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:24:28.029762 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.029735    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-dshm" (OuterVolumeSpecName: "dshm") pod "d8cc68de-6a98-4803-a8da-fe38424e897e" (UID: "d8cc68de-6a98-4803-a8da-fe38424e897e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:24:28.029881 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.029863    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8cc68de-6a98-4803-a8da-fe38424e897e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d8cc68de-6a98-4803-a8da-fe38424e897e" (UID: "d8cc68de-6a98-4803-a8da-fe38424e897e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:24:28.029925 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.029907    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8cc68de-6a98-4803-a8da-fe38424e897e-kube-api-access-4tw4h" (OuterVolumeSpecName: "kube-api-access-4tw4h") pod "d8cc68de-6a98-4803-a8da-fe38424e897e" (UID: "d8cc68de-6a98-4803-a8da-fe38424e897e"). InnerVolumeSpecName "kube-api-access-4tw4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:24:28.128842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.128747    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-dshm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:24:28.128842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.128779    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8cc68de-6a98-4803-a8da-fe38424e897e-tls-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:24:28.128842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.128793    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tw4h\" (UniqueName: \"kubernetes.io/projected/d8cc68de-6a98-4803-a8da-fe38424e897e-kube-api-access-4tw4h\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:24:28.128842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.128805    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8cc68de-6a98-4803-a8da-fe38424e897e-home\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:24:28.632331 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.632291    2575 generic.go:356] "Generic (PLEG): container finished" podID="d8cc68de-6a98-4803-a8da-fe38424e897e" containerID="9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be" exitCode=0
Jul 23 22:24:28.632752 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.632340    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"
Jul 23 22:24:28.632752 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.632378    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f" event={"ID":"d8cc68de-6a98-4803-a8da-fe38424e897e","Type":"ContainerDied","Data":"9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be"}
Jul 23 22:24:28.632752 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.632424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f" event={"ID":"d8cc68de-6a98-4803-a8da-fe38424e897e","Type":"ContainerDied","Data":"0f33f713ec6c555ae14e4f345736e160b127bbde923684471f8f50cfe75b55da"}
Jul 23 22:24:28.632752 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.632443    2575 scope.go:117] "RemoveContainer" containerID="9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be"
Jul 23 22:24:28.641818 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.641787    2575 scope.go:117] "RemoveContainer" containerID="9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be"
Jul 23 22:24:28.642082 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:24:28.642063    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be\": container with ID starting with 9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be not found: ID does not exist" containerID="9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be"
Jul 23 22:24:28.642147 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.642094    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be"} err="failed to get container status \"9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be\": rpc error: code = NotFound desc = could not find container \"9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be\": container with ID starting with 9b3fc0062cc909f5e12866b11e385da39d87d28703cdb60a079d2f95de5fa4be not found: ID does not exist"
Jul 23 22:24:28.650208 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.650174    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"]
Jul 23 22:24:28.654017 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:28.653996    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-0578d301/e2e-lifecycle-model-0578d301-kserve-5df7dd95c5-m769f"]
Jul 23 22:24:30.228338 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:30.228298    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8cc68de-6a98-4803-a8da-fe38424e897e" path="/var/lib/kubelet/pods/d8cc68de-6a98-4803-a8da-fe38424e897e/volumes"
Jul 23 22:24:30.851763 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:30.851732    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:36.683191 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:36.683153    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:36.783973 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:36.783938    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:49.687872 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:49.687836    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:24:49.845049 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:24:49.845011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:04.722020 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:04.721982    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:04.822080 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:04.822017    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:05.014851 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:05.014761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:05.246872 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:05.246825    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:06.371389 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:06.371329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:06.471026 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:06.470960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:06.571318 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:06.571288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:07.506466 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:07.506430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:07.657457 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:07.657417    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:09.209729 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:09.209690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:15.416060 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:15.416025    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:15.505757 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:15.505722    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:16.113732 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:16.113700    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:22.237125 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:22.237086    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:22.299913 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:22.299878    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:22.388717 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:22.388679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:22.484453 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:22.484420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:32.957139 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:32.957108    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:33.312843 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:33.312760    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:25:54.328062 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:25:54.327976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:00.438408 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:00.438353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:00.542807 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:00.542774    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:00.791177 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:00.791082    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:01.254013 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:01.253972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:01.546102 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:01.546007    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:04.647210 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:04.647173    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:04.738751 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:04.738718    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:04.873150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:04.873115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:05.275417 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:05.275380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:05.472644 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:05.472596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:05.569832 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:05.569757    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:09.171691 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:09.171653    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:09.526524 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:09.526430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:09.943650 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:09.943590    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:10.246775 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:10.246688    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:12.704901 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.704866    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:12.799032 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.798994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:12.935466 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.935421    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"]
Jul 23 22:26:12.936035 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.936006    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8cc68de-6a98-4803-a8da-fe38424e897e" containerName="main"
Jul 23 22:26:12.936035 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.936035    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8cc68de-6a98-4803-a8da-fe38424e897e" containerName="main"
Jul 23 22:26:12.936245 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.936158    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8cc68de-6a98-4803-a8da-fe38424e897e" containerName="main"
Jul 23 22:26:12.939786 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.939763    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:12.942232 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.942208    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b12a8c-serving-cert\""
Jul 23 22:26:12.942349 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.942210    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nz9df\""
Jul 23 22:26:12.949296 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.949271    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"]
Jul 23 22:26:12.987322 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:12.987245    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:13.078685 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.078605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.078866 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.078756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmfsb\" (UniqueName: \"kubernetes.io/projected/36714846-b5a1-45f7-9359-db315e6944a5-kube-api-access-cmfsb\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.078866 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.078808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36714846-b5a1-45f7-9359-db315e6944a5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.180208 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.180169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36714846-b5a1-45f7-9359-db315e6944a5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.180405 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.180239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.180405 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.180282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmfsb\" (UniqueName: \"kubernetes.io/projected/36714846-b5a1-45f7-9359-db315e6944a5-kube-api-access-cmfsb\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.180405 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:26:13.180334    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-serving-cert: secret "maas-api-e2e-inf-b-b12a8c-serving-cert" not found
Jul 23 22:26:13.180531 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:26:13.180409    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls podName:36714846-b5a1-45f7-9359-db315e6944a5 nodeName:}" failed. No retries permitted until 2026-07-23 22:26:13.680393337 +0000 UTC m=+2095.941393045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls") pod "maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" (UID: "36714846-b5a1-45f7-9359-db315e6944a5") : secret "maas-api-e2e-inf-b-b12a8c-serving-cert" not found
Jul 23 22:26:13.180873 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.180855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36714846-b5a1-45f7-9359-db315e6944a5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.188768 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.188743    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmfsb\" (UniqueName: \"kubernetes.io/projected/36714846-b5a1-45f7-9359-db315e6944a5-kube-api-access-cmfsb\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.442072 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.442020    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:13.542549 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.542505    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:13.685453 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:13.685425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:13.685650 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:26:13.685529    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-serving-cert: secret "maas-api-e2e-inf-b-b12a8c-serving-cert" not found
Jul 23 22:26:13.685650 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:26:13.685588    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls podName:36714846-b5a1-45f7-9359-db315e6944a5 nodeName:}" failed. No retries permitted until 2026-07-23 22:26:14.685575168 +0000 UTC m=+2096.946574874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls") pod "maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" (UID: "36714846-b5a1-45f7-9359-db315e6944a5") : secret "maas-api-e2e-inf-b-b12a8c-serving-cert" not found
Jul 23 22:26:14.007987 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:14.007948    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:14.695762 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:14.695717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:14.698120 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:14.698090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") pod \"maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:14.752023 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:14.751978    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:14.903481 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:14.903456    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"]
Jul 23 22:26:14.905179 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:26:14.905147    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36714846_b5a1_45f7_9359_db315e6944a5.slice/crio-9b3260ed5d7364b2774a97f2a87e53af5ec202760a751bdebb9f892da8314bb5 WatchSource:0}: Error finding container 9b3260ed5d7364b2774a97f2a87e53af5ec202760a751bdebb9f892da8314bb5: Status 404 returned error can't find the container with id 9b3260ed5d7364b2774a97f2a87e53af5ec202760a751bdebb9f892da8314bb5
Jul 23 22:26:14.906436 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:14.906417    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 22:26:15.258362 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:15.258324    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" event={"ID":"36714846-b5a1-45f7-9359-db315e6944a5","Type":"ContainerStarted","Data":"9b3260ed5d7364b2774a97f2a87e53af5ec202760a751bdebb9f892da8314bb5"}
Jul 23 22:26:15.712182 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:15.712097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:16.438585 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:16.438541    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:16.539896 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:16.539854    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:17.198240 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:17.198209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:17.273604 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:17.273516    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" event={"ID":"36714846-b5a1-45f7-9359-db315e6944a5","Type":"ContainerStarted","Data":"2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d"}
Jul 23 22:26:17.273797 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:17.273627    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:17.299249 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:17.299201    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" podStartSLOduration=3.181585571 podStartE2EDuration="5.299186258s" totalImagesPullingTime="2.117600687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:26:17.295787301 +0000 UTC m=+2099.556787031" watchObservedRunningTime="2026-07-23 22:26:17.299186258 +0000 UTC m=+2099.560186045"
Jul 23 22:26:18.436117 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:18.436088    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:26:18.442437 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:18.442403    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:26:18.825540 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:18.825465    2575 scope.go:117] "RemoveContainer" containerID="f3dfd88019a4427e399e218d718acc55f7f2450e19d6110ac504680766901875"
Jul 23 22:26:23.283685 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:23.283656    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:26:25.517110 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:25.517040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:26.379993 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:26.379953    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:26.481213 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:26.481175    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:34.253851 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:34.253816    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:34.353620 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:34.353586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:43.917358 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.917320    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"]
Jul 23 22:26:43.925745 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.925707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:43.929772 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.929744    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b12a8c\"/\"test-model-b12a8c-kserve-self-signed-certs\""
Jul 23 22:26:43.930021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.929745    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b12a8c\"/\"kube-root-ca.crt\""
Jul 23 22:26:43.930110 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.929751    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b12a8c\"/\"openshift-service-ca.crt\""
Jul 23 22:26:43.930445 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.930423    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b12a8c\"/\"default-dockercfg-8ckbc\""
Jul 23 22:26:43.939096 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.939068    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"]
Jul 23 22:26:43.976885 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:43.976847    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:26:44.057923 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.057883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-dshm\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.057923 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.057927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvdnq\" (UniqueName: \"kubernetes.io/projected/ebfa5043-8cbe-46dc-8d93-0f2b89457072-kube-api-access-hvdnq\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.058192 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.057968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-home\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.058192 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.058054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tls-certs\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.058192 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.058114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-model-cache\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.058192 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.058161    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tmp-dir\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159400 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-home\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159603 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tls-certs\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159603 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159476    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-model-cache\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159603 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tmp-dir\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159603 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-dshm\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159868 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvdnq\" (UniqueName: \"kubernetes.io/projected/ebfa5043-8cbe-46dc-8d93-0f2b89457072-kube-api-access-hvdnq\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159925 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159872    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-model-cache\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159975 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-home\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.159975 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.159951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tmp-dir\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.161785 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.161762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-dshm\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.162033 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.162017    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tls-certs\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.168561 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.168501    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvdnq\" (UniqueName: \"kubernetes.io/projected/ebfa5043-8cbe-46dc-8d93-0f2b89457072-kube-api-access-hvdnq\") pod \"test-model-b12a8c-kserve-84949c667f-k4cp4\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") " pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.238424 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.238379    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:44.390183 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.390141    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"]
Jul 23 22:26:44.393365 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:26:44.393336    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebfa5043_8cbe_46dc_8d93_0f2b89457072.slice/crio-e6c83bcee85ff000847b967c93f8eafab2ec065461364b4749dfc8ebeb9bd465 WatchSource:0}: Error finding container e6c83bcee85ff000847b967c93f8eafab2ec065461364b4749dfc8ebeb9bd465: Status 404 returned error can't find the container with id e6c83bcee85ff000847b967c93f8eafab2ec065461364b4749dfc8ebeb9bd465
Jul 23 22:26:44.430305 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:44.430237    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4" event={"ID":"ebfa5043-8cbe-46dc-8d93-0f2b89457072","Type":"ContainerStarted","Data":"e6c83bcee85ff000847b967c93f8eafab2ec065461364b4749dfc8ebeb9bd465"}
Jul 23 22:26:45.438349 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:45.438309    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4" event={"ID":"ebfa5043-8cbe-46dc-8d93-0f2b89457072","Type":"ContainerStarted","Data":"d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27"}
Jul 23 22:26:45.460138 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:45.460085    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4" podStartSLOduration=2.4600687470000002 podStartE2EDuration="2.460068747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:26:45.457194321 +0000 UTC m=+2127.718194049" watchObservedRunningTime="2026-07-23 22:26:45.460068747 +0000 UTC m=+2127.721068509"
Jul 23 22:26:54.239061 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:54.239025    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:54.239464 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:54.239097    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:54.251817 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:54.251793    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:26:54.500763 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:26:54.500683    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:27:26.882399 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:26.882362    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:27.731413 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:27.731374    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"]
Jul 23 22:27:27.750311 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:27.750260    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:28.434956 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:28.434913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:28.535093 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:28.535055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:38.667460 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:38.667421    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:42.753186 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:42.753132    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4" podUID="ebfa5043-8cbe-46dc-8d93-0f2b89457072" containerName="main" containerID="cri-o://d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27" gracePeriod=45
Jul 23 22:27:42.996111 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:42.996074    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:43.018034 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.017957    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:27:43.206023 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.205989    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tmp-dir\") pod \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") "
Jul 23 22:27:43.206196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206040    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-home\") pod \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") "
Jul 23 22:27:43.206196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206086    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvdnq\" (UniqueName: \"kubernetes.io/projected/ebfa5043-8cbe-46dc-8d93-0f2b89457072-kube-api-access-hvdnq\") pod \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") "
Jul 23 22:27:43.206196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206107    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tls-certs\") pod \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") "
Jul 23 22:27:43.206196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206128    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-dshm\") pod \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") "
Jul 23 22:27:43.206196 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206167    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-model-cache\") pod \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\" (UID: \"ebfa5043-8cbe-46dc-8d93-0f2b89457072\") "
Jul 23 22:27:43.206472 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206324    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ebfa5043-8cbe-46dc-8d93-0f2b89457072" (UID: "ebfa5043-8cbe-46dc-8d93-0f2b89457072"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:27:43.206472 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206363    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-home" (OuterVolumeSpecName: "home") pod "ebfa5043-8cbe-46dc-8d93-0f2b89457072" (UID: "ebfa5043-8cbe-46dc-8d93-0f2b89457072"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:27:43.206562 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206503    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-model-cache" (OuterVolumeSpecName: "model-cache") pod "ebfa5043-8cbe-46dc-8d93-0f2b89457072" (UID: "ebfa5043-8cbe-46dc-8d93-0f2b89457072"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:27:43.206562 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206521    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tmp-dir\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:43.206562 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.206543    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-home\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:43.208337 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.208314    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-dshm" (OuterVolumeSpecName: "dshm") pod "ebfa5043-8cbe-46dc-8d93-0f2b89457072" (UID: "ebfa5043-8cbe-46dc-8d93-0f2b89457072"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:27:43.208466 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.208447    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebfa5043-8cbe-46dc-8d93-0f2b89457072-kube-api-access-hvdnq" (OuterVolumeSpecName: "kube-api-access-hvdnq") pod "ebfa5043-8cbe-46dc-8d93-0f2b89457072" (UID: "ebfa5043-8cbe-46dc-8d93-0f2b89457072"). InnerVolumeSpecName "kube-api-access-hvdnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:27:43.208466 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.208454    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebfa5043-8cbe-46dc-8d93-0f2b89457072" (UID: "ebfa5043-8cbe-46dc-8d93-0f2b89457072"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:27:43.307605 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.307511    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvdnq\" (UniqueName: \"kubernetes.io/projected/ebfa5043-8cbe-46dc-8d93-0f2b89457072-kube-api-access-hvdnq\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:43.307605 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.307555    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfa5043-8cbe-46dc-8d93-0f2b89457072-tls-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:43.307605 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.307566    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-dshm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:43.307605 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.307574    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfa5043-8cbe-46dc-8d93-0f2b89457072-model-cache\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:43.390574 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.390535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:43.772247 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.772208    2575 generic.go:356] "Generic (PLEG): container finished" podID="ebfa5043-8cbe-46dc-8d93-0f2b89457072" containerID="d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27" exitCode=0
Jul 23 22:27:43.772247 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.772248    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"
Jul 23 22:27:43.772830 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.772288    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4" event={"ID":"ebfa5043-8cbe-46dc-8d93-0f2b89457072","Type":"ContainerDied","Data":"d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27"}
Jul 23 22:27:43.772830 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.772330    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4" event={"ID":"ebfa5043-8cbe-46dc-8d93-0f2b89457072","Type":"ContainerDied","Data":"e6c83bcee85ff000847b967c93f8eafab2ec065461364b4749dfc8ebeb9bd465"}
Jul 23 22:27:43.772830 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.772351    2575 scope.go:117] "RemoveContainer" containerID="d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27"
Jul 23 22:27:43.785957 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.785940    2575 scope.go:117] "RemoveContainer" containerID="d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27"
Jul 23 22:27:43.786227 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:27:43.786208    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27\": container with ID starting with d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27 not found: ID does not exist" containerID="d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27"
Jul 23 22:27:43.786307 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.786244    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27"} err="failed to get container status \"d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27\": rpc error: code = NotFound desc = could not find container \"d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27\": container with ID starting with d92c7a570bbd973fdd0640f468ad992a0ead5dee50c390c3216c780986035d27 not found: ID does not exist"
Jul 23 22:27:43.799091 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.799068    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"]
Jul 23 22:27:43.805121 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:43.805097    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b12a8c/test-model-b12a8c-kserve-84949c667f-k4cp4"]
Jul 23 22:27:44.227689 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:44.227653    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebfa5043-8cbe-46dc-8d93-0f2b89457072" path="/var/lib/kubelet/pods/ebfa5043-8cbe-46dc-8d93-0f2b89457072/volumes"
Jul 23 22:27:49.718130 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:49.718094    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:49.798118 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:49.798082    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:50.009455 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.009371    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:50.231105 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.231071    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:50.354859 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.354820    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"]
Jul 23 22:27:50.355238 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.355226    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfa5043-8cbe-46dc-8d93-0f2b89457072" containerName="main"
Jul 23 22:27:50.355294 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.355239    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfa5043-8cbe-46dc-8d93-0f2b89457072" containerName="main"
Jul 23 22:27:50.355336 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.355312    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfa5043-8cbe-46dc-8d93-0f2b89457072" containerName="main"
Jul 23 22:27:50.360153 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.360133    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:50.362724 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.362706    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:27:50.367225 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.367197    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"]
Jul 23 22:27:50.465777 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.465735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x\" (UID: \"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:50.566421 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.566385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x\" (UID: \"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:50.567025 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.567004    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x\" (UID: \"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:50.673087 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.672997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:50.814189 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.814165    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"]
Jul 23 22:27:50.816748 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:27:50.816719    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73ca08d_3674_441b_88e1_9cf4f8f9cbdb.slice/crio-8639df424f79945d98b88d40352d7e282ab6038189bf9c342ed33a52ad340c53 WatchSource:0}: Error finding container 8639df424f79945d98b88d40352d7e282ab6038189bf9c342ed33a52ad340c53: Status 404 returned error can't find the container with id 8639df424f79945d98b88d40352d7e282ab6038189bf9c342ed33a52ad340c53
Jul 23 22:27:50.889951 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.889917    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:50.989913 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:50.989828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:27:51.822562 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:51.822474    2575 generic.go:356] "Generic (PLEG): container finished" podID="a73ca08d-3674-441b-88e1-9cf4f8f9cbdb" containerID="c424899e6e61fbb1483bfc2a94873ffae52991c44e817c2059a03a3eeffc8f7a" exitCode=0
Jul 23 22:27:51.822964 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:51.822565    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x" event={"ID":"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb","Type":"ContainerDied","Data":"c424899e6e61fbb1483bfc2a94873ffae52991c44e817c2059a03a3eeffc8f7a"}
Jul 23 22:27:51.822964 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:51.822627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x" event={"ID":"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb","Type":"ContainerStarted","Data":"8639df424f79945d98b88d40352d7e282ab6038189bf9c342ed33a52ad340c53"}
Jul 23 22:27:52.974189 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:52.974162    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:52.989554 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:52.989521    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb-maas-api-service-ca\") pod \"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb\" (UID: \"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb\") "
Jul 23 22:27:52.989875 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:52.989853    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a73ca08d-3674-441b-88e1-9cf4f8f9cbdb" (UID: "a73ca08d-3674-441b-88e1-9cf4f8f9cbdb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:27:53.091216 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:53.091121    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb-maas-api-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:27:53.838362 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:53.838329    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"
Jul 23 22:27:53.838624 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:53.838325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x" event={"ID":"a73ca08d-3674-441b-88e1-9cf4f8f9cbdb","Type":"ContainerDied","Data":"8639df424f79945d98b88d40352d7e282ab6038189bf9c342ed33a52ad340c53"}
Jul 23 22:27:53.838624 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:27:53.838423    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8639df424f79945d98b88d40352d7e282ab6038189bf9c342ed33a52ad340c53"
Jul 23 22:28:00.904893 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:00.904853    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"]
Jul 23 22:28:00.905407 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:00.905118    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" podUID="36714846-b5a1-45f7-9359-db315e6944a5" containerName="maas-api" containerID="cri-o://2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d" gracePeriod=30
Jul 23 22:28:00.942019 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:00.941980    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:01.174390 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.174364    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:28:01.266363 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.266324    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") pod \"36714846-b5a1-45f7-9359-db315e6944a5\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") "
Jul 23 22:28:01.266549 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.266411    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36714846-b5a1-45f7-9359-db315e6944a5-openshift-service-ca\") pod \"36714846-b5a1-45f7-9359-db315e6944a5\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") "
Jul 23 22:28:01.266549 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.266450    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmfsb\" (UniqueName: \"kubernetes.io/projected/36714846-b5a1-45f7-9359-db315e6944a5-kube-api-access-cmfsb\") pod \"36714846-b5a1-45f7-9359-db315e6944a5\" (UID: \"36714846-b5a1-45f7-9359-db315e6944a5\") "
Jul 23 22:28:01.266849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.266809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36714846-b5a1-45f7-9359-db315e6944a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36714846-b5a1-45f7-9359-db315e6944a5" (UID: "36714846-b5a1-45f7-9359-db315e6944a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:28:01.268804 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.268772    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36714846-b5a1-45f7-9359-db315e6944a5" (UID: "36714846-b5a1-45f7-9359-db315e6944a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:28:01.268895 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.268828    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36714846-b5a1-45f7-9359-db315e6944a5-kube-api-access-cmfsb" (OuterVolumeSpecName: "kube-api-access-cmfsb") pod "36714846-b5a1-45f7-9359-db315e6944a5" (UID: "36714846-b5a1-45f7-9359-db315e6944a5"). InnerVolumeSpecName "kube-api-access-cmfsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:28:01.367687 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.367619    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36714846-b5a1-45f7-9359-db315e6944a5-maas-api-tls\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:28:01.367687 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.367682    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36714846-b5a1-45f7-9359-db315e6944a5-openshift-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:28:01.367687 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.367695    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmfsb\" (UniqueName: \"kubernetes.io/projected/36714846-b5a1-45f7-9359-db315e6944a5-kube-api-access-cmfsb\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:28:01.889711 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.889676    2575 generic.go:356] "Generic (PLEG): container finished" podID="36714846-b5a1-45f7-9359-db315e6944a5" containerID="2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d" exitCode=0
Jul 23 22:28:01.889881 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.889716    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"
Jul 23 22:28:01.889881 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.889749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" event={"ID":"36714846-b5a1-45f7-9359-db315e6944a5","Type":"ContainerDied","Data":"2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d"}
Jul 23 22:28:01.889881 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.889804    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq" event={"ID":"36714846-b5a1-45f7-9359-db315e6944a5","Type":"ContainerDied","Data":"9b3260ed5d7364b2774a97f2a87e53af5ec202760a751bdebb9f892da8314bb5"}
Jul 23 22:28:01.889881 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.889833    2575 scope.go:117] "RemoveContainer" containerID="2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d"
Jul 23 22:28:01.901849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.901824    2575 scope.go:117] "RemoveContainer" containerID="2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d"
Jul 23 22:28:01.902224 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:28:01.902201    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d\": container with ID starting with 2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d not found: ID does not exist" containerID="2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d"
Jul 23 22:28:01.902323 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.902231    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d"} err="failed to get container status \"2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d\": rpc error: code = NotFound desc = could not find container \"2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d\": container with ID starting with 2d29d3423a708bc66a27393a66802cbc5913672c9f942fa18d8cc016b976422d not found: ID does not exist"
Jul 23 22:28:01.915755 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.915725    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"]
Jul 23 22:28:01.919075 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:01.919052    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b12a8c-5ff66cc45-jdhpq"]
Jul 23 22:28:02.227891 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:02.227810    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36714846-b5a1-45f7-9359-db315e6944a5" path="/var/lib/kubelet/pods/36714846-b5a1-45f7-9359-db315e6944a5/volumes"
Jul 23 22:28:05.434894 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:05.434845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:05.453425 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:05.453391    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"]
Jul 23 22:28:05.461758 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:05.461713    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b12a8c-brz4x"]
Jul 23 22:28:05.848447 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:05.848399    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:06.227112 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:06.227041    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73ca08d-3674-441b-88e1-9cf4f8f9cbdb" path="/var/lib/kubelet/pods/a73ca08d-3674-441b-88e1-9cf4f8f9cbdb/volumes"
Jul 23 22:28:11.792693 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:11.792656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:11.893611 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:11.893574    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:12.082755 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:12.082621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:12.322897 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:12.322865    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:12.681620 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:12.681586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:13.014797 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:13.014713    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:13.642285 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:13.642249    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:16.331114 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:16.331071    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:16.429577 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:16.429535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:16.523577 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:16.523534    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:16.672394 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:16.672362    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:17.140397 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:17.140365    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:17.242674 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:17.242605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:20.873447 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:20.873410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:21.392048 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:21.392005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:21.693915 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:21.693809    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:21.793892 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:21.793852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:24.365012 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.364968    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:24.478434 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.478397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:24.635311 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635227    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"]
Jul 23 22:28:24.635727 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635703    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36714846-b5a1-45f7-9359-db315e6944a5" containerName="maas-api"
Jul 23 22:28:24.635822 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635728    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36714846-b5a1-45f7-9359-db315e6944a5" containerName="maas-api"
Jul 23 22:28:24.635822 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635774    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73ca08d-3674-441b-88e1-9cf4f8f9cbdb" containerName="revoke-keys"
Jul 23 22:28:24.635822 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635783    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73ca08d-3674-441b-88e1-9cf4f8f9cbdb" containerName="revoke-keys"
Jul 23 22:28:24.635984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635882    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a73ca08d-3674-441b-88e1-9cf4f8f9cbdb" containerName="revoke-keys"
Jul 23 22:28:24.635984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.635898    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="36714846-b5a1-45f7-9359-db315e6944a5" containerName="maas-api"
Jul 23 22:28:24.639035 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.639015    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.641781 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.641758    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9684d1-serving-cert\""
Jul 23 22:28:24.642048 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.642027    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nz9df\""
Jul 23 22:28:24.653418 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.653385    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"]
Jul 23 22:28:24.682170 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.682136    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:24.780651 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.780589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12758784-e746-43f6-bb3a-1cfd9f93ea2a-openshift-service-ca\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.780818 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.780654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5nrc\" (UniqueName: \"kubernetes.io/projected/12758784-e746-43f6-bb3a-1cfd9f93ea2a-kube-api-access-r5nrc\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.780818 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.780695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12758784-e746-43f6-bb3a-1cfd9f93ea2a-maas-api-tls\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.882212 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.882173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12758784-e746-43f6-bb3a-1cfd9f93ea2a-openshift-service-ca\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.882212 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.882214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nrc\" (UniqueName: \"kubernetes.io/projected/12758784-e746-43f6-bb3a-1cfd9f93ea2a-kube-api-access-r5nrc\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.882431 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.882241    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12758784-e746-43f6-bb3a-1cfd9f93ea2a-maas-api-tls\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.882883 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.882861    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12758784-e746-43f6-bb3a-1cfd9f93ea2a-openshift-service-ca\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.884710 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.884681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12758784-e746-43f6-bb3a-1cfd9f93ea2a-maas-api-tls\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.890333 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.890273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5nrc\" (UniqueName: \"kubernetes.io/projected/12758784-e746-43f6-bb3a-1cfd9f93ea2a-kube-api-access-r5nrc\") pod \"maas-api-e2e-api-b-9684d1-657689bfbf-klqrx\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:24.919236 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.919208    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:24.950829 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:24.950801    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:25.106742 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:25.106714    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"]
Jul 23 22:28:25.108566 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:28:25.108530    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12758784_e746_43f6_bb3a_1cfd9f93ea2a.slice/crio-cbe76b081fc4a05c02eff5f9afa46e6a3d3c239fb34d48fe8781ea8de8649c4f WatchSource:0}: Error finding container cbe76b081fc4a05c02eff5f9afa46e6a3d3c239fb34d48fe8781ea8de8649c4f: Status 404 returned error can't find the container with id cbe76b081fc4a05c02eff5f9afa46e6a3d3c239fb34d48fe8781ea8de8649c4f
Jul 23 22:28:25.277113 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:25.277011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:25.377128 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:25.377087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:26.034478 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:26.034436    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx" event={"ID":"12758784-e746-43f6-bb3a-1cfd9f93ea2a","Type":"ContainerStarted","Data":"f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428"}
Jul 23 22:28:26.034678 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:26.034662    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:26.034739 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:26.034708    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx" event={"ID":"12758784-e746-43f6-bb3a-1cfd9f93ea2a","Type":"ContainerStarted","Data":"cbe76b081fc4a05c02eff5f9afa46e6a3d3c239fb34d48fe8781ea8de8649c4f"}
Jul 23 22:28:26.056120 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:26.056071    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx" podStartSLOduration=1.6728043860000001 podStartE2EDuration="2.056056538s" totalImagesPullingTime="383.252152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:28:26.054457126 +0000 UTC m=+2228.315456854" watchObservedRunningTime="2026-07-23 22:28:26.056056538 +0000 UTC m=+2228.317056266"
Jul 23 22:28:28.901834 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:28.901799    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:32.046650 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:32.046609    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:28:37.128577 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.128546    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"]
Jul 23 22:28:37.132991 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.132970    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:37.135165 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.135145    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:28:37.142436 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.142410    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"]
Jul 23 22:28:37.298021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.297978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dcb1ba-75d3-434f-97ce-57834f376798-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6\" (UID: \"77dcb1ba-75d3-434f-97ce-57834f376798\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:37.399057 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.398964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dcb1ba-75d3-434f-97ce-57834f376798-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6\" (UID: \"77dcb1ba-75d3-434f-97ce-57834f376798\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:37.399619 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.399598    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dcb1ba-75d3-434f-97ce-57834f376798-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6\" (UID: \"77dcb1ba-75d3-434f-97ce-57834f376798\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:37.445968 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.445927    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:37.792865 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.792837    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"]
Jul 23 22:28:37.794705 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:28:37.794667    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77dcb1ba_75d3_434f_97ce_57834f376798.slice/crio-c98305222d4a7e38ef4f4b7e90645076923ee7f6efab04c2bb64fdef8d39a96e WatchSource:0}: Error finding container c98305222d4a7e38ef4f4b7e90645076923ee7f6efab04c2bb64fdef8d39a96e: Status 404 returned error can't find the container with id c98305222d4a7e38ef4f4b7e90645076923ee7f6efab04c2bb64fdef8d39a96e
Jul 23 22:28:37.807494 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.807457    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:37.911450 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:37.911417    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:38.108916 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:38.108884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6" event={"ID":"77dcb1ba-75d3-434f-97ce-57834f376798","Type":"ContainerStarted","Data":"c98305222d4a7e38ef4f4b7e90645076923ee7f6efab04c2bb64fdef8d39a96e"}
Jul 23 22:28:39.116842 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:39.116807    2575 generic.go:356] "Generic (PLEG): container finished" podID="77dcb1ba-75d3-434f-97ce-57834f376798" containerID="ab0933f2c640f05209d55508e846f3843c850c47e52dc9782309b22ef827c5d0" exitCode=0
Jul 23 22:28:39.117218 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:39.116881    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6" event={"ID":"77dcb1ba-75d3-434f-97ce-57834f376798","Type":"ContainerDied","Data":"ab0933f2c640f05209d55508e846f3843c850c47e52dc9782309b22ef827c5d0"}
Jul 23 22:28:40.268103 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:40.268079    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:40.423009 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:40.422880    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dcb1ba-75d3-434f-97ce-57834f376798-maas-api-service-ca\") pod \"77dcb1ba-75d3-434f-97ce-57834f376798\" (UID: \"77dcb1ba-75d3-434f-97ce-57834f376798\") "
Jul 23 22:28:40.423271 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:40.423247    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77dcb1ba-75d3-434f-97ce-57834f376798-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77dcb1ba-75d3-434f-97ce-57834f376798" (UID: "77dcb1ba-75d3-434f-97ce-57834f376798"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:28:40.523476 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:40.523439    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77dcb1ba-75d3-434f-97ce-57834f376798-maas-api-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:28:41.129462 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:41.129429    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"
Jul 23 22:28:41.129684 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:41.129428    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6" event={"ID":"77dcb1ba-75d3-434f-97ce-57834f376798","Type":"ContainerDied","Data":"c98305222d4a7e38ef4f4b7e90645076923ee7f6efab04c2bb64fdef8d39a96e"}
Jul 23 22:28:41.129684 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:41.129561    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c98305222d4a7e38ef4f4b7e90645076923ee7f6efab04c2bb64fdef8d39a96e"
Jul 23 22:28:46.029443 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:46.029407    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:46.128874 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:46.128840    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:47.668686 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:47.668642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:47.774056 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:47.774000    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:52.936000 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:52.935315    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:57.202085 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:57.202051    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:57.208420 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:57.208386    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"]
Jul 23 22:28:57.211413 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:57.211391    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-83cd2f-86vf6"]
Jul 23 22:28:57.599702 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:57.599669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:28:58.228492 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:28:58.228450    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77dcb1ba-75d3-434f-97ce-57834f376798" path="/var/lib/kubelet/pods/77dcb1ba-75d3-434f-97ce-57834f376798/volumes"
Jul 23 22:29:03.261848 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.261812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:03.889200 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.889165    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"]
Jul 23 22:29:03.889577 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.889564    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77dcb1ba-75d3-434f-97ce-57834f376798" containerName="revoke-keys"
Jul 23 22:29:03.889622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.889579    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="77dcb1ba-75d3-434f-97ce-57834f376798" containerName="revoke-keys"
Jul 23 22:29:03.889684 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.889659    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="77dcb1ba-75d3-434f-97ce-57834f376798" containerName="revoke-keys"
Jul 23 22:29:03.892923 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.892905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:03.895276 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.895249    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:29:03.901263 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:03.901240    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"]
Jul 23 22:29:04.041526 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:04.041489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb1e422-1ea8-4d72-af29-3800e51fe4e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh\" (UID: \"9bb1e422-1ea8-4d72-af29-3800e51fe4e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:04.142489 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:04.142398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb1e422-1ea8-4d72-af29-3800e51fe4e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh\" (UID: \"9bb1e422-1ea8-4d72-af29-3800e51fe4e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:04.143016 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:04.142996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb1e422-1ea8-4d72-af29-3800e51fe4e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh\" (UID: \"9bb1e422-1ea8-4d72-af29-3800e51fe4e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:04.205643 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:04.205609    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:04.350907 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:04.350881    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"]
Jul 23 22:29:04.353500 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:29:04.353474    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb1e422_1ea8_4d72_af29_3800e51fe4e6.slice/crio-82def1ecb0af8c5ca3f77b06102d7035c1573db859fc2a9b5207c4e4d7786f87 WatchSource:0}: Error finding container 82def1ecb0af8c5ca3f77b06102d7035c1573db859fc2a9b5207c4e4d7786f87: Status 404 returned error can't find the container with id 82def1ecb0af8c5ca3f77b06102d7035c1573db859fc2a9b5207c4e4d7786f87
Jul 23 22:29:04.663281 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:04.663214    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:05.272358 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:05.272319    2575 generic.go:356] "Generic (PLEG): container finished" podID="9bb1e422-1ea8-4d72-af29-3800e51fe4e6" containerID="8c246fa083e553221ce9423c334f10c262fe547e4166524b5ff57fe0348c772e" exitCode=0
Jul 23 22:29:05.272527 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:05.272410    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh" event={"ID":"9bb1e422-1ea8-4d72-af29-3800e51fe4e6","Type":"ContainerDied","Data":"8c246fa083e553221ce9423c334f10c262fe547e4166524b5ff57fe0348c772e"}
Jul 23 22:29:05.272527 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:05.272464    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh" event={"ID":"9bb1e422-1ea8-4d72-af29-3800e51fe4e6","Type":"ContainerStarted","Data":"82def1ecb0af8c5ca3f77b06102d7035c1573db859fc2a9b5207c4e4d7786f87"}
Jul 23 22:29:06.420572 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:06.420546    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:06.563971 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:06.563884    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb1e422-1ea8-4d72-af29-3800e51fe4e6-maas-api-service-ca\") pod \"9bb1e422-1ea8-4d72-af29-3800e51fe4e6\" (UID: \"9bb1e422-1ea8-4d72-af29-3800e51fe4e6\") "
Jul 23 22:29:06.564240 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:06.564215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bb1e422-1ea8-4d72-af29-3800e51fe4e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bb1e422-1ea8-4d72-af29-3800e51fe4e6" (UID: "9bb1e422-1ea8-4d72-af29-3800e51fe4e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:29:06.664472 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:06.664439    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb1e422-1ea8-4d72-af29-3800e51fe4e6-maas-api-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:29:07.284823 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:07.284786    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"
Jul 23 22:29:07.284823 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:07.284813    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh" event={"ID":"9bb1e422-1ea8-4d72-af29-3800e51fe4e6","Type":"ContainerDied","Data":"82def1ecb0af8c5ca3f77b06102d7035c1573db859fc2a9b5207c4e4d7786f87"}
Jul 23 22:29:07.285039 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:07.284862    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82def1ecb0af8c5ca3f77b06102d7035c1573db859fc2a9b5207c4e4d7786f87"
Jul 23 22:29:08.198314 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:08.198266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:19.512808 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.512744    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"]
Jul 23 22:29:19.513377 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.513229    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx" podUID="12758784-e746-43f6-bb3a-1cfd9f93ea2a" containerName="maas-api" containerID="cri-o://f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428" gracePeriod=30
Jul 23 22:29:19.541874 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.541846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:19.778511 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.778452    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:29:19.885516 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.885481    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12758784-e746-43f6-bb3a-1cfd9f93ea2a-maas-api-tls\") pod \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") "
Jul 23 22:29:19.885738 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.885565    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12758784-e746-43f6-bb3a-1cfd9f93ea2a-openshift-service-ca\") pod \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") "
Jul 23 22:29:19.885738 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.885688    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5nrc\" (UniqueName: \"kubernetes.io/projected/12758784-e746-43f6-bb3a-1cfd9f93ea2a-kube-api-access-r5nrc\") pod \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\" (UID: \"12758784-e746-43f6-bb3a-1cfd9f93ea2a\") "
Jul 23 22:29:19.886026 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.885987    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12758784-e746-43f6-bb3a-1cfd9f93ea2a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "12758784-e746-43f6-bb3a-1cfd9f93ea2a" (UID: "12758784-e746-43f6-bb3a-1cfd9f93ea2a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:29:19.887657 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.887613    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12758784-e746-43f6-bb3a-1cfd9f93ea2a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "12758784-e746-43f6-bb3a-1cfd9f93ea2a" (UID: "12758784-e746-43f6-bb3a-1cfd9f93ea2a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:29:19.887733 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.887687    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12758784-e746-43f6-bb3a-1cfd9f93ea2a-kube-api-access-r5nrc" (OuterVolumeSpecName: "kube-api-access-r5nrc") pod "12758784-e746-43f6-bb3a-1cfd9f93ea2a" (UID: "12758784-e746-43f6-bb3a-1cfd9f93ea2a"). InnerVolumeSpecName "kube-api-access-r5nrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:29:19.986820 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.986781    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5nrc\" (UniqueName: \"kubernetes.io/projected/12758784-e746-43f6-bb3a-1cfd9f93ea2a-kube-api-access-r5nrc\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:29:19.986820 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.986814    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12758784-e746-43f6-bb3a-1cfd9f93ea2a-maas-api-tls\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:29:19.986820 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:19.986823    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12758784-e746-43f6-bb3a-1cfd9f93ea2a-openshift-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:29:20.363598 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.363564    2575 generic.go:356] "Generic (PLEG): container finished" podID="12758784-e746-43f6-bb3a-1cfd9f93ea2a" containerID="f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428" exitCode=0
Jul 23 22:29:20.363802 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.363609    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"
Jul 23 22:29:20.363802 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.363662    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx" event={"ID":"12758784-e746-43f6-bb3a-1cfd9f93ea2a","Type":"ContainerDied","Data":"f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428"}
Jul 23 22:29:20.363802 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.363719    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx" event={"ID":"12758784-e746-43f6-bb3a-1cfd9f93ea2a","Type":"ContainerDied","Data":"cbe76b081fc4a05c02eff5f9afa46e6a3d3c239fb34d48fe8781ea8de8649c4f"}
Jul 23 22:29:20.363802 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.363744    2575 scope.go:117] "RemoveContainer" containerID="f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428"
Jul 23 22:29:20.375353 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.375326    2575 scope.go:117] "RemoveContainer" containerID="f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428"
Jul 23 22:29:20.375852 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:29:20.375829    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428\": container with ID starting with f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428 not found: ID does not exist" containerID="f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428"
Jul 23 22:29:20.375927 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.375865    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428"} err="failed to get container status \"f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428\": rpc error: code = NotFound desc = could not find container \"f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428\": container with ID starting with f2d09715a65d76f5214618dfbd699f1f196a79e60cf8eca5bfcb5d5e44b90428 not found: ID does not exist"
Jul 23 22:29:20.381924 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.381898    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"]
Jul 23 22:29:20.386456 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:20.386428    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9684d1-657689bfbf-klqrx"]
Jul 23 22:29:22.228008 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:22.227970    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12758784-e746-43f6-bb3a-1cfd9f93ea2a" path="/var/lib/kubelet/pods/12758784-e746-43f6-bb3a-1cfd9f93ea2a/volumes"
Jul 23 22:29:23.967097 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:23.967064    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:23.971774 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:23.971747    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"]
Jul 23 22:29:23.975181 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:23.975153    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9684d1-vbqwh"]
Jul 23 22:29:24.227690 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:24.227604    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bb1e422-1ea8-4d72-af29-3800e51fe4e6" path="/var/lib/kubelet/pods/9bb1e422-1ea8-4d72-af29-3800e51fe4e6/volumes"
Jul 23 22:29:24.385501 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:24.385466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:30.527265 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:30.527228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:30.627144 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:30.627110    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:30.811847 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:30.811763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:31.077529 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:31.077449    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:31.438857 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:31.438812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:31.791645 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:31.791549    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:32.171974 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:32.171944    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:34.738192 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:34.738157    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:34.842113 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:34.842079    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:34.938203 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:34.938171    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:35.041378 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:35.041293    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:35.440303 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:35.440271    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:35.539820 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:35.539767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:37.474720 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:37.474685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:37.575556 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:37.575517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:39.273085 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:39.273050    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:39.634230 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:39.634179    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:39.973071 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:39.972990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:40.072990 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:40.072958    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:42.795930 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.795894    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:42.890915 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.890880    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"]
Jul 23 22:29:42.891406 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891384    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb1e422-1ea8-4d72-af29-3800e51fe4e6" containerName="revoke-keys"
Jul 23 22:29:42.891406 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891407    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb1e422-1ea8-4d72-af29-3800e51fe4e6" containerName="revoke-keys"
Jul 23 22:29:42.891596 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891428    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12758784-e746-43f6-bb3a-1cfd9f93ea2a" containerName="maas-api"
Jul 23 22:29:42.891596 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891435    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="12758784-e746-43f6-bb3a-1cfd9f93ea2a" containerName="maas-api"
Jul 23 22:29:42.891596 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891518    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb1e422-1ea8-4d72-af29-3800e51fe4e6" containerName="revoke-keys"
Jul 23 22:29:42.891596 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891536    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="12758784-e746-43f6-bb3a-1cfd9f93ea2a" containerName="maas-api"
Jul 23 22:29:42.891843 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.891674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:42.895937 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.895916    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:42.898312 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.898290    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-6bf8de-serving-cert\""
Jul 23 22:29:42.898481 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.898463    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nz9df\""
Jul 23 22:29:42.907496 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.907472    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"]
Jul 23 22:29:42.965459 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.965430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:42.986541 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.986516    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bbj\" (UniqueName: \"kubernetes.io/projected/a5d8b969-e544-47ae-b914-01f1d945af42-kube-api-access-g5bbj\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:42.986731 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.986572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5d8b969-e544-47ae-b914-01f1d945af42-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:42.986731 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:42.986618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.088004 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.087914    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bbj\" (UniqueName: \"kubernetes.io/projected/a5d8b969-e544-47ae-b914-01f1d945af42-kube-api-access-g5bbj\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.088004 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.087978    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5d8b969-e544-47ae-b914-01f1d945af42-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.088004 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.088003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.088230 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:29:43.088103    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-serving-cert: secret "maas-api-e2e-shared-b-6bf8de-serving-cert" not found
Jul 23 22:29:43.088230 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:29:43.088161    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls podName:a5d8b969-e544-47ae-b914-01f1d945af42 nodeName:}" failed. No retries permitted until 2026-07-23 22:29:43.588142946 +0000 UTC m=+2305.849142652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls") pod "maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" (UID: "a5d8b969-e544-47ae-b914-01f1d945af42") : secret "maas-api-e2e-shared-b-6bf8de-serving-cert" not found
Jul 23 22:29:43.088615 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.088594    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5d8b969-e544-47ae-b914-01f1d945af42-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.097787 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.097754    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bbj\" (UniqueName: \"kubernetes.io/projected/a5d8b969-e544-47ae-b914-01f1d945af42-kube-api-access-g5bbj\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.337806 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.337771    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:43.469952 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.469919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:43.571788 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.571751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:43.593034 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.593006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.595373 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.595344    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls\") pod \"maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.807857 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.807770    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:43.949135 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:43.949110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"]
Jul 23 22:29:43.950621 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:29:43.950592    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d8b969_e544_47ae_b914_01f1d945af42.slice/crio-61f81bdedd26f02f5599a1d394497d5afe045c25dbe182820e0035cf67ef6ea0 WatchSource:0}: Error finding container 61f81bdedd26f02f5599a1d394497d5afe045c25dbe182820e0035cf67ef6ea0: Status 404 returned error can't find the container with id 61f81bdedd26f02f5599a1d394497d5afe045c25dbe182820e0035cf67ef6ea0
Jul 23 22:29:44.503077 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:44.503034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" event={"ID":"a5d8b969-e544-47ae-b914-01f1d945af42","Type":"ContainerStarted","Data":"ea2489c451552d40f00ee283989ae0c958b3944332d3b9825fc87caeb09567b4"}
Jul 23 22:29:44.503252 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:44.503100    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" event={"ID":"a5d8b969-e544-47ae-b914-01f1d945af42","Type":"ContainerStarted","Data":"61f81bdedd26f02f5599a1d394497d5afe045c25dbe182820e0035cf67ef6ea0"}
Jul 23 22:29:44.503252 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:44.503152    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:44.520006 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:44.519954    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" podStartSLOduration=2.154817573 podStartE2EDuration="2.519938537s" totalImagesPullingTime="365.120964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:29:44.518795765 +0000 UTC m=+2306.779795505" watchObservedRunningTime="2026-07-23 22:29:44.519938537 +0000 UTC m=+2306.780938265"
Jul 23 22:29:47.335431 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:47.335401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:50.514831 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:50.514801    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:29:56.207723 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:56.207683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:29:56.308712 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:29:56.308675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:30:00.140336 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.140295    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"]
Jul 23 22:30:00.144027 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.143999    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"
Jul 23 22:30:00.144428 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.144412    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"]
Jul 23 22:30:00.148181 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.148155    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:30:00.155764 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.153236    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"]
Jul 23 22:30:00.155764 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.153528    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"
Jul 23 22:30:00.163446 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.163418    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"
Jul 23 22:30:00.163712 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.163695    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"
Jul 23 22:30:00.164291 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.164273    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"]
Jul 23 22:30:00.168312 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.168289    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"]
Jul 23 22:30:00.172539 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.172517    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"]
Jul 23 22:30:00.173572 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.173555    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"
Jul 23 22:30:00.213667 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.213615    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"
Jul 23 22:30:00.376481 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.376451    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"]
Jul 23 22:30:00.378936 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:30:00.378906    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3643d543_8857_4890_9412_999f33812dd0.slice/crio-051bd749319fc730cb0d039fe2229b9bb8d8ecc6398de0b9025c32ccf3a57e35 WatchSource:0}: Error finding container 051bd749319fc730cb0d039fe2229b9bb8d8ecc6398de0b9025c32ccf3a57e35: Status 404 returned error can't find the container with id 051bd749319fc730cb0d039fe2229b9bb8d8ecc6398de0b9025c32ccf3a57e35
Jul 23 22:30:00.403925 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.403899    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"]
Jul 23 22:30:00.409837 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:30:00.409814    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51238466_af83_4830_bbdd_f99b964e6cd9.slice/crio-d2b922dc74be24fb6f43ba70a2502a0f9a1036d95ac953be5a567a6f24eb2dff WatchSource:0}: Error finding container d2b922dc74be24fb6f43ba70a2502a0f9a1036d95ac953be5a567a6f24eb2dff: Status 404 returned error can't find the container with id d2b922dc74be24fb6f43ba70a2502a0f9a1036d95ac953be5a567a6f24eb2dff
Jul 23 22:30:00.553316 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.553285    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"]
Jul 23 22:30:00.555024 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:30:00.554992    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc5c474_d5bd_4af1_b59a_c1b4923dbae9.slice/crio-67750de82467a4c2727582c649fe9652f5c59aeaecabba82c2c65db27542feb1 WatchSource:0}: Error finding container 67750de82467a4c2727582c649fe9652f5c59aeaecabba82c2c65db27542feb1: Status 404 returned error can't find the container with id 67750de82467a4c2727582c649fe9652f5c59aeaecabba82c2c65db27542feb1
Jul 23 22:30:00.598300 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.598272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2" event={"ID":"51238466-af83-4830-bbdd-f99b964e6cd9","Type":"ContainerStarted","Data":"d2b922dc74be24fb6f43ba70a2502a0f9a1036d95ac953be5a567a6f24eb2dff"}
Jul 23 22:30:00.599487 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.599464    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj" event={"ID":"3643d543-8857-4890-9412-999f33812dd0","Type":"ContainerStarted","Data":"051bd749319fc730cb0d039fe2229b9bb8d8ecc6398de0b9025c32ccf3a57e35"}
Jul 23 22:30:00.600548 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:00.600527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v" event={"ID":"6bc5c474-d5bd-4af1-b59a-c1b4923dbae9","Type":"ContainerStarted","Data":"67750de82467a4c2727582c649fe9652f5c59aeaecabba82c2c65db27542feb1"}
Jul 23 22:30:01.608692 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:01.608653    2575 generic.go:356] "Generic (PLEG): container finished" podID="3643d543-8857-4890-9412-999f33812dd0" containerID="36afe9566945df43f6bd7c0e114ff2b0d9970ef11e3187617ce43907efd86ecd" exitCode=0
Jul 23 22:30:01.609118 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:01.608752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj" event={"ID":"3643d543-8857-4890-9412-999f33812dd0","Type":"ContainerDied","Data":"36afe9566945df43f6bd7c0e114ff2b0d9970ef11e3187617ce43907efd86ecd"}
Jul 23 22:30:01.610414 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:01.610389    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v" event={"ID":"6bc5c474-d5bd-4af1-b59a-c1b4923dbae9","Type":"ContainerStarted","Data":"5cb8e5505ee18c0363dfe040a05e053361c0c1b768b3fd64b80cab189f952324"}
Jul 23 22:30:01.611941 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:01.611919    2575 generic.go:356] "Generic (PLEG): container finished" podID="51238466-af83-4830-bbdd-f99b964e6cd9" containerID="4e99d16e85c002232a9dc0a5d15cd30631248f69c96b8a447e3633db78bbcd33" exitCode=0
Jul 23 22:30:01.612051 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:01.611951    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2" event={"ID":"51238466-af83-4830-bbdd-f99b964e6cd9","Type":"ContainerDied","Data":"4e99d16e85c002232a9dc0a5d15cd30631248f69c96b8a447e3633db78bbcd33"}
Jul 23 22:30:01.639909 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:01.639863    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v" podStartSLOduration=1.639845537 podStartE2EDuration="1.639845537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:30:01.637704614 +0000 UTC m=+2323.898704360" watchObservedRunningTime="2026-07-23 22:30:01.639845537 +0000 UTC m=+2323.900845265"
Jul 23 22:30:02.622971 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:02.622939    2575 generic.go:356] "Generic (PLEG): container finished" podID="6bc5c474-d5bd-4af1-b59a-c1b4923dbae9" containerID="5cb8e5505ee18c0363dfe040a05e053361c0c1b768b3fd64b80cab189f952324" exitCode=0
Jul 23 22:30:02.623453 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:02.622999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v" event={"ID":"6bc5c474-d5bd-4af1-b59a-c1b4923dbae9","Type":"ContainerDied","Data":"5cb8e5505ee18c0363dfe040a05e053361c0c1b768b3fd64b80cab189f952324"}
Jul 23 22:30:02.791461 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:02.791434    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"
Jul 23 22:30:02.818407 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:02.818381    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"
Jul 23 22:30:03.638397 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.638363    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2"
Jul 23 22:30:03.638841 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.638391    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-8nww2" event={"ID":"51238466-af83-4830-bbdd-f99b964e6cd9","Type":"ContainerDied","Data":"d2b922dc74be24fb6f43ba70a2502a0f9a1036d95ac953be5a567a6f24eb2dff"}
Jul 23 22:30:03.638841 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.638470    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2b922dc74be24fb6f43ba70a2502a0f9a1036d95ac953be5a567a6f24eb2dff"
Jul 23 22:30:03.639824 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.639797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj" event={"ID":"3643d543-8857-4890-9412-999f33812dd0","Type":"ContainerDied","Data":"051bd749319fc730cb0d039fe2229b9bb8d8ecc6398de0b9025c32ccf3a57e35"}
Jul 23 22:30:03.639949 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.639846    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="051bd749319fc730cb0d039fe2229b9bb8d8ecc6398de0b9025c32ccf3a57e35"
Jul 23 22:30:03.639949 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.639909    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"
Jul 23 22:30:03.784082 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:03.784050    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"
Jul 23 22:30:04.270045 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.270013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:30:04.369691 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.369656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:30:04.382358 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382319    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"]
Jul 23 22:30:04.382839 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382820    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3643d543-8857-4890-9412-999f33812dd0" containerName="cleanup"
Jul 23 22:30:04.382937 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382841    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3643d543-8857-4890-9412-999f33812dd0" containerName="cleanup"
Jul 23 22:30:04.382937 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382854    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc5c474-d5bd-4af1-b59a-c1b4923dbae9" containerName="cleanup"
Jul 23 22:30:04.382937 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382863    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc5c474-d5bd-4af1-b59a-c1b4923dbae9" containerName="cleanup"
Jul 23 22:30:04.382937 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382895    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51238466-af83-4830-bbdd-f99b964e6cd9" containerName="cleanup"
Jul 23 22:30:04.382937 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.382905    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51238466-af83-4830-bbdd-f99b964e6cd9" containerName="cleanup"
Jul 23 22:30:04.383179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.383012    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc5c474-d5bd-4af1-b59a-c1b4923dbae9" containerName="cleanup"
Jul 23 22:30:04.383179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.383026    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="51238466-af83-4830-bbdd-f99b964e6cd9" containerName="cleanup"
Jul 23 22:30:04.383179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.383036    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3643d543-8857-4890-9412-999f33812dd0" containerName="cleanup"
Jul 23 22:30:04.386220 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.386199    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.388797 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.388772    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1c273d\"/\"openshift-service-ca.crt\""
Jul 23 22:30:04.388921 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.388814    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1c273d\"/\"kube-root-ca.crt\""
Jul 23 22:30:04.389694 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.389675    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1c273d\"/\"default-dockercfg-6nccm\""
Jul 23 22:30:04.389906 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.389892    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1c273d\"/\"auth-test-model-1c273d-kserve-self-signed-certs\""
Jul 23 22:30:04.397109 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.397088    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"]
Jul 23 22:30:04.493069 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.492991    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:30:04.502007 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.501985    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa4ebff-843b-4ef5-a281-e7448848edf6-tls-certs\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.502088 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.502040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-home\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.502088 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.502061    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-dshm\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.502179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.502136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-model-cache\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.502223 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.502192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-tmp-dir\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.502264 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.502221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv4s4\" (UniqueName: \"kubernetes.io/projected/cfa4ebff-843b-4ef5-a281-e7448848edf6-kube-api-access-wv4s4\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603081 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-tmp-dir\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603081 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603094    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv4s4\" (UniqueName: \"kubernetes.io/projected/cfa4ebff-843b-4ef5-a281-e7448848edf6-kube-api-access-wv4s4\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603328 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa4ebff-843b-4ef5-a281-e7448848edf6-tls-certs\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603328 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-home\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603328 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-dshm\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603328 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-model-cache\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603540 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603515    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-tmp-dir\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603603 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603545    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-home\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.603681 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.603671    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-model-cache\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.605452 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.605427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-dshm\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.605792 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.605771    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa4ebff-843b-4ef5-a281-e7448848edf6-tls-certs\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.610601 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.610579    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv4s4\" (UniqueName: \"kubernetes.io/projected/cfa4ebff-843b-4ef5-a281-e7448848edf6-kube-api-access-wv4s4\") pod \"auth-test-model-1c273d-kserve-66b4b6df79-9h4ph\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") " pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.648756 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.648719    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v" event={"ID":"6bc5c474-d5bd-4af1-b59a-c1b4923dbae9","Type":"ContainerDied","Data":"67750de82467a4c2727582c649fe9652f5c59aeaecabba82c2c65db27542feb1"}
Jul 23 22:30:04.648756 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.648753    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"
Jul 23 22:30:04.649164 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.648771    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67750de82467a4c2727582c649fe9652f5c59aeaecabba82c2c65db27542feb1"
Jul 23 22:30:04.695746 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.695711    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:04.858395 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:04.858360    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"]
Jul 23 22:30:04.859429 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:30:04.859400    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfa4ebff_843b_4ef5_a281_e7448848edf6.slice/crio-29c9d16c7bbefc3189f99f93d43f028df4c4e76410533490fbe11b3e27b1cbd0 WatchSource:0}: Error finding container 29c9d16c7bbefc3189f99f93d43f028df4c4e76410533490fbe11b3e27b1cbd0: Status 404 returned error can't find the container with id 29c9d16c7bbefc3189f99f93d43f028df4c4e76410533490fbe11b3e27b1cbd0
Jul 23 22:30:05.658096 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:05.658052    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph" event={"ID":"cfa4ebff-843b-4ef5-a281-e7448848edf6","Type":"ContainerStarted","Data":"d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28"}
Jul 23 22:30:05.658497 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:05.658107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph" event={"ID":"cfa4ebff-843b-4ef5-a281-e7448848edf6","Type":"ContainerStarted","Data":"29c9d16c7bbefc3189f99f93d43f028df4c4e76410533490fbe11b3e27b1cbd0"}
Jul 23 22:30:05.676898 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:05.676845    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph" podStartSLOduration=1.6768312060000001 podStartE2EDuration="1.676831206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:30:05.674587621 +0000 UTC m=+2327.935587360" watchObservedRunningTime="2026-07-23 22:30:05.676831206 +0000 UTC m=+2327.937830933"
Jul 23 22:30:14.696367 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:14.696271    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:14.696367 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:14.696351    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:14.708972 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:14.708944    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:14.725091 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:14.725061    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:30:31.016454 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:30:31.016415    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:31:18.476627 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:31:18.476590    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:31:18.483033 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:31:18.483010    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:31:35.950434 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:31:35.950393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:32:11.273741 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:11.273707    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:32:30.708801 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:30.708767    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"]
Jul 23 22:32:30.737104 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:30.737068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:32:31.610408 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:31.610374    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:32:39.285596 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:39.285559    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:32:45.729991 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:45.729920    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph" podUID="cfa4ebff-843b-4ef5-a281-e7448848edf6" containerName="main" containerID="cri-o://d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28" gracePeriod=45
Jul 23 22:32:45.991697 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:45.991609    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:32:46.136232 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.136196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:32:46.166409 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166371    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-tmp-dir\") pod \"cfa4ebff-843b-4ef5-a281-e7448848edf6\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") "
Jul 23 22:32:46.166592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166449    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-dshm\") pod \"cfa4ebff-843b-4ef5-a281-e7448848edf6\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") "
Jul 23 22:32:46.166592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166477    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv4s4\" (UniqueName: \"kubernetes.io/projected/cfa4ebff-843b-4ef5-a281-e7448848edf6-kube-api-access-wv4s4\") pod \"cfa4ebff-843b-4ef5-a281-e7448848edf6\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") "
Jul 23 22:32:46.166592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166560    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-model-cache\") pod \"cfa4ebff-843b-4ef5-a281-e7448848edf6\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") "
Jul 23 22:32:46.166592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166586    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa4ebff-843b-4ef5-a281-e7448848edf6-tls-certs\") pod \"cfa4ebff-843b-4ef5-a281-e7448848edf6\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") "
Jul 23 22:32:46.166864 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166612    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-home\") pod \"cfa4ebff-843b-4ef5-a281-e7448848edf6\" (UID: \"cfa4ebff-843b-4ef5-a281-e7448848edf6\") "
Jul 23 22:32:46.166864 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166623    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfa4ebff-843b-4ef5-a281-e7448848edf6" (UID: "cfa4ebff-843b-4ef5-a281-e7448848edf6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:32:46.166979 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166857    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfa4ebff-843b-4ef5-a281-e7448848edf6" (UID: "cfa4ebff-843b-4ef5-a281-e7448848edf6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:32:46.166979 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.166903    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-home" (OuterVolumeSpecName: "home") pod "cfa4ebff-843b-4ef5-a281-e7448848edf6" (UID: "cfa4ebff-843b-4ef5-a281-e7448848edf6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:32:46.167080 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.167011    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-tmp-dir\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:32:46.167080 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.167026    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-model-cache\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:32:46.167080 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.167035    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-home\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:32:46.168868 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.168841    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-dshm" (OuterVolumeSpecName: "dshm") pod "cfa4ebff-843b-4ef5-a281-e7448848edf6" (UID: "cfa4ebff-843b-4ef5-a281-e7448848edf6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:32:46.168996 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.168982    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa4ebff-843b-4ef5-a281-e7448848edf6-kube-api-access-wv4s4" (OuterVolumeSpecName: "kube-api-access-wv4s4") pod "cfa4ebff-843b-4ef5-a281-e7448848edf6" (UID: "cfa4ebff-843b-4ef5-a281-e7448848edf6"). InnerVolumeSpecName "kube-api-access-wv4s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:32:46.169039 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.169006    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa4ebff-843b-4ef5-a281-e7448848edf6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfa4ebff-843b-4ef5-a281-e7448848edf6" (UID: "cfa4ebff-843b-4ef5-a281-e7448848edf6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:32:46.268323 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.268238    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfa4ebff-843b-4ef5-a281-e7448848edf6-tls-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:32:46.268323 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.268271    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfa4ebff-843b-4ef5-a281-e7448848edf6-dshm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:32:46.268323 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.268281    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv4s4\" (UniqueName: \"kubernetes.io/projected/cfa4ebff-843b-4ef5-a281-e7448848edf6-kube-api-access-wv4s4\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:32:46.653301 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.653259    2575 generic.go:356] "Generic (PLEG): container finished" podID="cfa4ebff-843b-4ef5-a281-e7448848edf6" containerID="d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28" exitCode=0
Jul 23 22:32:46.653503 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.653348    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"
Jul 23 22:32:46.653503 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.653375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph" event={"ID":"cfa4ebff-843b-4ef5-a281-e7448848edf6","Type":"ContainerDied","Data":"d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28"}
Jul 23 22:32:46.653503 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.653425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph" event={"ID":"cfa4ebff-843b-4ef5-a281-e7448848edf6","Type":"ContainerDied","Data":"29c9d16c7bbefc3189f99f93d43f028df4c4e76410533490fbe11b3e27b1cbd0"}
Jul 23 22:32:46.653503 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.653447    2575 scope.go:117] "RemoveContainer" containerID="d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28"
Jul 23 22:32:46.664954 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.664925    2575 scope.go:117] "RemoveContainer" containerID="d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28"
Jul 23 22:32:46.665244 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:32:46.665220    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28\": container with ID starting with d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28 not found: ID does not exist" containerID="d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28"
Jul 23 22:32:46.665302 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.665253    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28"} err="failed to get container status \"d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28\": rpc error: code = NotFound desc = could not find container \"d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28\": container with ID starting with d5099b8b24e0f3782348c0b28f8da7ff077e825866566d106988c985df128a28 not found: ID does not exist"
Jul 23 22:32:46.669871 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.669846    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"]
Jul 23 22:32:46.673248 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:46.673182    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1c273d/auth-test-model-1c273d-kserve-66b4b6df79-9h4ph"]
Jul 23 22:32:48.227323 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:32:48.227283    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfa4ebff-843b-4ef5-a281-e7448848edf6" path="/var/lib/kubelet/pods/cfa4ebff-843b-4ef5-a281-e7448848edf6/volumes"
Jul 23 22:33:24.299233 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.299196    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"]
Jul 23 22:33:24.299756 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.299680    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa4ebff-843b-4ef5-a281-e7448848edf6" containerName="main"
Jul 23 22:33:24.299756 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.299695    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa4ebff-843b-4ef5-a281-e7448848edf6" containerName="main"
Jul 23 22:33:24.299838 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.299787    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa4ebff-843b-4ef5-a281-e7448848edf6" containerName="main"
Jul 23 22:33:24.304209 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.304192    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.307344 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.307319    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"kube-root-ca.crt\""
Jul 23 22:33:24.307479 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.307318    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"sub-test-model-6bf8de-kserve-self-signed-certs\""
Jul 23 22:33:24.307479 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.307321    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"openshift-service-ca.crt\""
Jul 23 22:33:24.307479 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.307377    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"default-dockercfg-dgdl5\""
Jul 23 22:33:24.313594 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.313571    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"]
Jul 23 22:33:24.389869 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.389838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:33:24.402096 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.402059    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhgln\" (UniqueName: \"kubernetes.io/projected/47499d12-f646-4d4b-84aa-c84a3ed196bf-kube-api-access-bhgln\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.402258 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.402109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-tmp-dir\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.402258 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.402132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-dshm\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.402258 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.402209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-home\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.402258 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.402227    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-model-cache\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.402397 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.402300    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47499d12-f646-4d4b-84aa-c84a3ed196bf-tls-certs\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503326 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47499d12-f646-4d4b-84aa-c84a3ed196bf-tls-certs\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503514 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503343    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhgln\" (UniqueName: \"kubernetes.io/projected/47499d12-f646-4d4b-84aa-c84a3ed196bf-kube-api-access-bhgln\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503514 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503369    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-tmp-dir\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503514 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-dshm\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503514 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-home\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503514 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-model-cache\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503863 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503835    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-tmp-dir\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503997 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-model-cache\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.503997 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.503944    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-home\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.505753 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.505733    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-dshm\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.505919 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.505904    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47499d12-f646-4d4b-84aa-c84a3ed196bf-tls-certs\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.512535 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.512516    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhgln\" (UniqueName: \"kubernetes.io/projected/47499d12-f646-4d4b-84aa-c84a3ed196bf-kube-api-access-bhgln\") pod \"sub-test-model-6bf8de-kserve-786cc8464f-wwkxh\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") " pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.614872 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.614829    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:24.769819 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.769778    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"]
Jul 23 22:33:24.775398 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:33:24.775365    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47499d12_f646_4d4b_84aa_c84a3ed196bf.slice/crio-7e69f4935b8a35781cefd6e673996b4da435c669d5472b1e985bdbedc17991f8 WatchSource:0}: Error finding container 7e69f4935b8a35781cefd6e673996b4da435c669d5472b1e985bdbedc17991f8: Status 404 returned error can't find the container with id 7e69f4935b8a35781cefd6e673996b4da435c669d5472b1e985bdbedc17991f8
Jul 23 22:33:24.884042 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.883956    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh" event={"ID":"47499d12-f646-4d4b-84aa-c84a3ed196bf","Type":"ContainerStarted","Data":"3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2"}
Jul 23 22:33:24.884042 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.884009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh" event={"ID":"47499d12-f646-4d4b-84aa-c84a3ed196bf","Type":"ContainerStarted","Data":"7e69f4935b8a35781cefd6e673996b4da435c669d5472b1e985bdbedc17991f8"}
Jul 23 22:33:24.903408 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:24.903354    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh" podStartSLOduration=0.903338514 podStartE2EDuration="903.338514ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:33:24.900793007 +0000 UTC m=+2527.161792736" watchObservedRunningTime="2026-07-23 22:33:24.903338514 +0000 UTC m=+2527.164338241"
Jul 23 22:33:34.615727 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:34.615691    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:34.616234 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:34.615745    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:34.628584 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:34.628553    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:34.956291 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:34.956203    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:33:55.637414 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:55.637378    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:33:55.737828 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:33:55.737797    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:07.384613 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:07.384564    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:08.289136 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:08.289101    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"]
Jul 23 22:34:08.580159 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:08.580055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:11.423934 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:11.423896    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:18.944643 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:18.944606    2575 scope.go:117] "RemoveContainer" containerID="c424899e6e61fbb1483bfc2a94873ffae52991c44e817c2059a03a3eeffc8f7a"
Jul 23 22:34:23.309802 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.309742    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh" podUID="47499d12-f646-4d4b-84aa-c84a3ed196bf" containerName="main" containerID="cri-o://3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2" gracePeriod=45
Jul 23 22:34:23.571673 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.571600    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:34:23.687447 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687415    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhgln\" (UniqueName: \"kubernetes.io/projected/47499d12-f646-4d4b-84aa-c84a3ed196bf-kube-api-access-bhgln\") pod \"47499d12-f646-4d4b-84aa-c84a3ed196bf\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") "
Jul 23 22:34:23.687622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687489    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-dshm\") pod \"47499d12-f646-4d4b-84aa-c84a3ed196bf\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") "
Jul 23 22:34:23.687622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687535    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-home\") pod \"47499d12-f646-4d4b-84aa-c84a3ed196bf\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") "
Jul 23 22:34:23.687622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687561    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47499d12-f646-4d4b-84aa-c84a3ed196bf-tls-certs\") pod \"47499d12-f646-4d4b-84aa-c84a3ed196bf\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") "
Jul 23 22:34:23.687622 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687590    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-model-cache\") pod \"47499d12-f646-4d4b-84aa-c84a3ed196bf\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") "
Jul 23 22:34:23.687846 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687666    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-tmp-dir\") pod \"47499d12-f646-4d4b-84aa-c84a3ed196bf\" (UID: \"47499d12-f646-4d4b-84aa-c84a3ed196bf\") "
Jul 23 22:34:23.687846 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.687782    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-home" (OuterVolumeSpecName: "home") pod "47499d12-f646-4d4b-84aa-c84a3ed196bf" (UID: "47499d12-f646-4d4b-84aa-c84a3ed196bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:34:23.688042 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.688008    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "47499d12-f646-4d4b-84aa-c84a3ed196bf" (UID: "47499d12-f646-4d4b-84aa-c84a3ed196bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:34:23.688166 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.688022    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-home\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:34:23.688166 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.688024    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "47499d12-f646-4d4b-84aa-c84a3ed196bf" (UID: "47499d12-f646-4d4b-84aa-c84a3ed196bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:34:23.689887 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.689861    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47499d12-f646-4d4b-84aa-c84a3ed196bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "47499d12-f646-4d4b-84aa-c84a3ed196bf" (UID: "47499d12-f646-4d4b-84aa-c84a3ed196bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:34:23.689988 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.689898    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-dshm" (OuterVolumeSpecName: "dshm") pod "47499d12-f646-4d4b-84aa-c84a3ed196bf" (UID: "47499d12-f646-4d4b-84aa-c84a3ed196bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:34:23.690047 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.689986    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47499d12-f646-4d4b-84aa-c84a3ed196bf-kube-api-access-bhgln" (OuterVolumeSpecName: "kube-api-access-bhgln") pod "47499d12-f646-4d4b-84aa-c84a3ed196bf" (UID: "47499d12-f646-4d4b-84aa-c84a3ed196bf"). InnerVolumeSpecName "kube-api-access-bhgln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:34:23.788984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.788942    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47499d12-f646-4d4b-84aa-c84a3ed196bf-tls-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:34:23.788984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.788975    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-model-cache\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:34:23.788984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.788983    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-tmp-dir\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:34:23.788984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.788993    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhgln\" (UniqueName: \"kubernetes.io/projected/47499d12-f646-4d4b-84aa-c84a3ed196bf-kube-api-access-bhgln\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:34:23.789248 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:23.789002    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47499d12-f646-4d4b-84aa-c84a3ed196bf-dshm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:34:24.239575 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.239543    2575 generic.go:356] "Generic (PLEG): container finished" podID="47499d12-f646-4d4b-84aa-c84a3ed196bf" containerID="3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2" exitCode=0
Jul 23 22:34:24.239734 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.239578    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"
Jul 23 22:34:24.239734 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.239618    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh" event={"ID":"47499d12-f646-4d4b-84aa-c84a3ed196bf","Type":"ContainerDied","Data":"3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2"}
Jul 23 22:34:24.239734 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.239672    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh" event={"ID":"47499d12-f646-4d4b-84aa-c84a3ed196bf","Type":"ContainerDied","Data":"7e69f4935b8a35781cefd6e673996b4da435c669d5472b1e985bdbedc17991f8"}
Jul 23 22:34:24.239734 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.239693    2575 scope.go:117] "RemoveContainer" containerID="3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2"
Jul 23 22:34:24.249080 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.249058    2575 scope.go:117] "RemoveContainer" containerID="3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2"
Jul 23 22:34:24.249362 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:34:24.249341    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2\": container with ID starting with 3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2 not found: ID does not exist" containerID="3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2"
Jul 23 22:34:24.249431 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.249370    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2"} err="failed to get container status \"3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2\": rpc error: code = NotFound desc = could not find container \"3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2\": container with ID starting with 3e9dc783754af1faf663424f00a8aab3870a44a2524b47456203a7f3340b12d2 not found: ID does not exist"
Jul 23 22:34:24.257045 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.257014    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"]
Jul 23 22:34:24.261649 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:24.261607    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/sub-test-model-6bf8de-kserve-786cc8464f-wwkxh"]
Jul 23 22:34:26.227914 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:26.227866    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47499d12-f646-4d4b-84aa-c84a3ed196bf" path="/var/lib/kubelet/pods/47499d12-f646-4d4b-84aa-c84a3ed196bf/volumes"
Jul 23 22:34:38.369433 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:38.369388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:38.470944 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:38.470905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:47.103585 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:47.103546    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:50.673912 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.673873    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"]
Jul 23 22:34:50.674476 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.674456    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47499d12-f646-4d4b-84aa-c84a3ed196bf" containerName="main"
Jul 23 22:34:50.674563 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.674479    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="47499d12-f646-4d4b-84aa-c84a3ed196bf" containerName="main"
Jul 23 22:34:50.674618 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.674585    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="47499d12-f646-4d4b-84aa-c84a3ed196bf" containerName="main"
Jul 23 22:34:50.678075 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.678052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.680766 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.680745    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"openshift-service-ca.crt\""
Jul 23 22:34:50.680906 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.680885    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"kube-root-ca.crt\""
Jul 23 22:34:50.680983 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.680940    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"default-dockercfg-dgdl5\""
Jul 23 22:34:50.680983 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.680945    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6bf8de\"/\"rate-test-model-6bf8de-kserve-self-signed-certs\""
Jul 23 22:34:50.688745 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.688721    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"]
Jul 23 22:34:50.699972 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.699943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:50.845976 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.845935    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-home\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.846150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.846012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28339bb2-d7a4-48e3-8041-6d034bd73caa-tls-certs\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.846150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.846028    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwvwk\" (UniqueName: \"kubernetes.io/projected/28339bb2-d7a4-48e3-8041-6d034bd73caa-kube-api-access-qwvwk\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.846150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.846058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-dshm\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.846150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.846074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-tmp-dir\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.846311 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.846148    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-model-cache\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947438 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28339bb2-d7a4-48e3-8041-6d034bd73caa-tls-certs\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947438 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwvwk\" (UniqueName: \"kubernetes.io/projected/28339bb2-d7a4-48e3-8041-6d034bd73caa-kube-api-access-qwvwk\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947438 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947430    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-dshm\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947779 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947453    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-tmp-dir\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947779 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947493    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-model-cache\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947779 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-home\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.947980 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.947961    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-tmp-dir\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.948057 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.948000    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-model-cache\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.948115 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.948067    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-home\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.950002 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.949979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-dshm\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.950259 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.950244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28339bb2-d7a4-48e3-8041-6d034bd73caa-tls-certs\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.955148 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.955127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwvwk\" (UniqueName: \"kubernetes.io/projected/28339bb2-d7a4-48e3-8041-6d034bd73caa-kube-api-access-qwvwk\") pod \"rate-test-model-6bf8de-kserve-57588666dc-kjbf9\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") " pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:50.988957 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:50.988928    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:34:51.133916 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:51.133887    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"]
Jul 23 22:34:51.135020 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:34:51.134995    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28339bb2_d7a4_48e3_8041_6d034bd73caa.slice/crio-f266b051fa5f525834a80b206b190320b596dacd6115e91b46b82cadb89cae6b WatchSource:0}: Error finding container f266b051fa5f525834a80b206b190320b596dacd6115e91b46b82cadb89cae6b: Status 404 returned error can't find the container with id f266b051fa5f525834a80b206b190320b596dacd6115e91b46b82cadb89cae6b
Jul 23 22:34:51.398383 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:51.398344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9" event={"ID":"28339bb2-d7a4-48e3-8041-6d034bd73caa","Type":"ContainerStarted","Data":"bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c"}
Jul 23 22:34:51.398383 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:51.398392    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9" event={"ID":"28339bb2-d7a4-48e3-8041-6d034bd73caa","Type":"ContainerStarted","Data":"f266b051fa5f525834a80b206b190320b596dacd6115e91b46b82cadb89cae6b"}
Jul 23 22:34:51.416188 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:51.416137    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9" podStartSLOduration=1.416123907 podStartE2EDuration="1.416123907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:34:51.415262687 +0000 UTC m=+2613.676262418" watchObservedRunningTime="2026-07-23 22:34:51.416123907 +0000 UTC m=+2613.677123635"
Jul 23 22:34:57.104872 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:57.104835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:34:57.205745 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:34:57.205683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:00.989338 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:00.989301    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:35:00.989716 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:00.989355    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:35:01.001585 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:01.001560    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:35:01.472965 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:01.472932    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:35:05.054487 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:05.054453    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:05.165450 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:05.165408    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:18.968329 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:18.968296    2575 scope.go:117] "RemoveContainer" containerID="8c246fa083e553221ce9423c334f10c262fe547e4166524b5ff57fe0348c772e"
Jul 23 22:35:18.977554 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:18.977535    2575 scope.go:117] "RemoveContainer" containerID="ab0933f2c640f05209d55508e846f3843c850c47e52dc9782309b22ef827c5d0"
Jul 23 22:35:35.650882 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:35.650837    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:36.665506 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:36.665469    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"]
Jul 23 22:35:36.686057 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:36.686010    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:37.683755 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.683722    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140371672283952"]
Jul 23 22:35:37.688135 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.688114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:37.690723 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.690698    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rmhjm\""
Jul 23 22:35:37.693685 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.693662    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140371672283952"]
Jul 23 22:35:37.782762 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.782727    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hbll\" (UniqueName: \"kubernetes.io/projected/90d1db49-4c47-4107-a1ae-4abfd576f926-kube-api-access-6hbll\") pod \"test-curl-7464-140371672283952\" (UID: \"90d1db49-4c47-4107-a1ae-4abfd576f926\") " pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:37.883918 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.883881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hbll\" (UniqueName: \"kubernetes.io/projected/90d1db49-4c47-4107-a1ae-4abfd576f926-kube-api-access-6hbll\") pod \"test-curl-7464-140371672283952\" (UID: \"90d1db49-4c47-4107-a1ae-4abfd576f926\") " pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:37.892335 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.892312    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hbll\" (UniqueName: \"kubernetes.io/projected/90d1db49-4c47-4107-a1ae-4abfd576f926-kube-api-access-6hbll\") pod \"test-curl-7464-140371672283952\" (UID: \"90d1db49-4c47-4107-a1ae-4abfd576f926\") " pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:37.999516 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:37.999421    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:38.141456 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:38.141421    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140371672283952"]
Jul 23 22:35:38.142214 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:35:38.142185    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90d1db49_4c47_4107_a1ae_4abfd576f926.slice/crio-cf636ab46b480a94c0139940fd1839752ff74ba477b0db127f3ddb61ba00e467 WatchSource:0}: Error finding container cf636ab46b480a94c0139940fd1839752ff74ba477b0db127f3ddb61ba00e467: Status 404 returned error can't find the container with id cf636ab46b480a94c0139940fd1839752ff74ba477b0db127f3ddb61ba00e467
Jul 23 22:35:38.143928 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:38.143908    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 22:35:38.686958 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:38.686923    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140371672283952" event={"ID":"90d1db49-4c47-4107-a1ae-4abfd576f926","Type":"ContainerStarted","Data":"cf636ab46b480a94c0139940fd1839752ff74ba477b0db127f3ddb61ba00e467"}
Jul 23 22:35:39.695548 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:39.695511    2575 generic.go:356] "Generic (PLEG): container finished" podID="90d1db49-4c47-4107-a1ae-4abfd576f926" containerID="dbd91c0202ef502823ed5c3d0d3944a5c0a3cecce6fa11f5615dd185287d614b" exitCode=0
Jul 23 22:35:39.695933 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:39.695580    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140371672283952" event={"ID":"90d1db49-4c47-4107-a1ae-4abfd576f926","Type":"ContainerDied","Data":"dbd91c0202ef502823ed5c3d0d3944a5c0a3cecce6fa11f5615dd185287d614b"}
Jul 23 22:35:40.841942 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:40.841915    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:40.865150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:40.865123    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140371672283952_90d1db49-4c47-4107-a1ae-4abfd576f926/test-curl-7464-140371672283952/0.log"
Jul 23 22:35:40.888354 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:40.888305    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140371672283952"]
Jul 23 22:35:40.898375 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:40.898342    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140371672283952"]
Jul 23 22:35:40.908565 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:40.908537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hbll\" (UniqueName: \"kubernetes.io/projected/90d1db49-4c47-4107-a1ae-4abfd576f926-kube-api-access-6hbll\") pod \"90d1db49-4c47-4107-a1ae-4abfd576f926\" (UID: \"90d1db49-4c47-4107-a1ae-4abfd576f926\") "
Jul 23 22:35:40.910693 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:40.910671    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90d1db49-4c47-4107-a1ae-4abfd576f926-kube-api-access-6hbll" (OuterVolumeSpecName: "kube-api-access-6hbll") pod "90d1db49-4c47-4107-a1ae-4abfd576f926" (UID: "90d1db49-4c47-4107-a1ae-4abfd576f926"). InnerVolumeSpecName "kube-api-access-6hbll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:35:41.010000 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.009915    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hbll\" (UniqueName: \"kubernetes.io/projected/90d1db49-4c47-4107-a1ae-4abfd576f926-kube-api-access-6hbll\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:41.022145 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.022110    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140371670992944"]
Jul 23 22:35:41.022557 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.022541    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90d1db49-4c47-4107-a1ae-4abfd576f926" containerName="test-curl-7464-140371672283952"
Jul 23 22:35:41.022600 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.022560    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="90d1db49-4c47-4107-a1ae-4abfd576f926" containerName="test-curl-7464-140371672283952"
Jul 23 22:35:41.022730 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.022716    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="90d1db49-4c47-4107-a1ae-4abfd576f926" containerName="test-curl-7464-140371672283952"
Jul 23 22:35:41.025983 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.025959    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:41.036111 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.036079    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140371670992944"]
Jul 23 22:35:41.111308 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.111265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmt6j\" (UniqueName: \"kubernetes.io/projected/4e57551b-c805-4768-ad73-327ea72ff12f-kube-api-access-rmt6j\") pod \"test-curl-7464-140371670992944\" (UID: \"4e57551b-c805-4768-ad73-327ea72ff12f\") " pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:41.212186 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.212148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmt6j\" (UniqueName: \"kubernetes.io/projected/4e57551b-c805-4768-ad73-327ea72ff12f-kube-api-access-rmt6j\") pod \"test-curl-7464-140371670992944\" (UID: \"4e57551b-c805-4768-ad73-327ea72ff12f\") " pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:41.220188 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.220162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmt6j\" (UniqueName: \"kubernetes.io/projected/4e57551b-c805-4768-ad73-327ea72ff12f-kube-api-access-rmt6j\") pod \"test-curl-7464-140371670992944\" (UID: \"4e57551b-c805-4768-ad73-327ea72ff12f\") " pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:41.339586 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.339465    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:41.483673 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.483642    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140371670992944"]
Jul 23 22:35:41.484995 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:35:41.484965    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e57551b_c805_4768_ad73_327ea72ff12f.slice/crio-ce6eef8cec22ce21e8b6b5848275196508fba474dcf8dba15cc6b03fb89fe8f3 WatchSource:0}: Error finding container ce6eef8cec22ce21e8b6b5848275196508fba474dcf8dba15cc6b03fb89fe8f3: Status 404 returned error can't find the container with id ce6eef8cec22ce21e8b6b5848275196508fba474dcf8dba15cc6b03fb89fe8f3
Jul 23 22:35:41.709350 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.709312    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140371670992944" event={"ID":"4e57551b-c805-4768-ad73-327ea72ff12f","Type":"ContainerStarted","Data":"ce6eef8cec22ce21e8b6b5848275196508fba474dcf8dba15cc6b03fb89fe8f3"}
Jul 23 22:35:41.710561 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.710536    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:41.710717 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:41.710578    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf636ab46b480a94c0139940fd1839752ff74ba477b0db127f3ddb61ba00e467"
Jul 23 22:35:41.721369 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:35:41.721336    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-140371672283952\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="90d1db49-4c47-4107-a1ae-4abfd576f926" pod="opendatahub/test-curl-7464-140371672283952"
Jul 23 22:35:42.228356 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:42.228303    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90d1db49-4c47-4107-a1ae-4abfd576f926" path="/var/lib/kubelet/pods/90d1db49-4c47-4107-a1ae-4abfd576f926/volumes"
Jul 23 22:35:42.718165 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:42.718122    2575 generic.go:356] "Generic (PLEG): container finished" podID="4e57551b-c805-4768-ad73-327ea72ff12f" containerID="4073b4906b6d7b5a516fea9e2f330363e48e3b14a39612a45ab7123f5248ad91" exitCode=0
Jul 23 22:35:42.718343 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:42.718207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140371670992944" event={"ID":"4e57551b-c805-4768-ad73-327ea72ff12f","Type":"ContainerDied","Data":"4073b4906b6d7b5a516fea9e2f330363e48e3b14a39612a45ab7123f5248ad91"}
Jul 23 22:35:43.878389 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:43.878365    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:43.900768 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:43.900714    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140371670992944_4e57551b-c805-4768-ad73-327ea72ff12f/test-curl-7464-140371670992944/0.log"
Jul 23 22:35:43.927141 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:43.927107    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140371670992944"]
Jul 23 22:35:43.932332 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:43.932303    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140371670992944"]
Jul 23 22:35:43.938716 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:43.938689    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmt6j\" (UniqueName: \"kubernetes.io/projected/4e57551b-c805-4768-ad73-327ea72ff12f-kube-api-access-rmt6j\") pod \"4e57551b-c805-4768-ad73-327ea72ff12f\" (UID: \"4e57551b-c805-4768-ad73-327ea72ff12f\") "
Jul 23 22:35:43.940715 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:43.940689    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e57551b-c805-4768-ad73-327ea72ff12f-kube-api-access-rmt6j" (OuterVolumeSpecName: "kube-api-access-rmt6j") pod "4e57551b-c805-4768-ad73-327ea72ff12f" (UID: "4e57551b-c805-4768-ad73-327ea72ff12f"). InnerVolumeSpecName "kube-api-access-rmt6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:35:44.039621 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:44.039536    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmt6j\" (UniqueName: \"kubernetes.io/projected/4e57551b-c805-4768-ad73-327ea72ff12f-kube-api-access-rmt6j\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:44.228077 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:44.228033    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e57551b-c805-4768-ad73-327ea72ff12f" path="/var/lib/kubelet/pods/4e57551b-c805-4768-ad73-327ea72ff12f/volumes"
Jul 23 22:35:44.732356 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:44.732320    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371670992944"
Jul 23 22:35:44.732554 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:44.732360    2575 scope.go:117] "RemoveContainer" containerID="4073b4906b6d7b5a516fea9e2f330363e48e3b14a39612a45ab7123f5248ad91"
Jul 23 22:35:51.687582 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:51.687526    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9" podUID="28339bb2-d7a4-48e3-8041-6d034bd73caa" containerName="main" containerID="cri-o://bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c" gracePeriod=45
Jul 23 22:35:51.944447 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:51.944387    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:35:52.013195 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013164    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-dshm\") pod \"28339bb2-d7a4-48e3-8041-6d034bd73caa\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") "
Jul 23 22:35:52.013378 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013213    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28339bb2-d7a4-48e3-8041-6d034bd73caa-tls-certs\") pod \"28339bb2-d7a4-48e3-8041-6d034bd73caa\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") "
Jul 23 22:35:52.013378 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013290    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwvwk\" (UniqueName: \"kubernetes.io/projected/28339bb2-d7a4-48e3-8041-6d034bd73caa-kube-api-access-qwvwk\") pod \"28339bb2-d7a4-48e3-8041-6d034bd73caa\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") "
Jul 23 22:35:52.013378 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013320    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-model-cache\") pod \"28339bb2-d7a4-48e3-8041-6d034bd73caa\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") "
Jul 23 22:35:52.013378 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013362    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-tmp-dir\") pod \"28339bb2-d7a4-48e3-8041-6d034bd73caa\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") "
Jul 23 22:35:52.013587 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013417    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-home\") pod \"28339bb2-d7a4-48e3-8041-6d034bd73caa\" (UID: \"28339bb2-d7a4-48e3-8041-6d034bd73caa\") "
Jul 23 22:35:52.013665 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013615    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-model-cache" (OuterVolumeSpecName: "model-cache") pod "28339bb2-d7a4-48e3-8041-6d034bd73caa" (UID: "28339bb2-d7a4-48e3-8041-6d034bd73caa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:35:52.013665 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013610    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28339bb2-d7a4-48e3-8041-6d034bd73caa" (UID: "28339bb2-d7a4-48e3-8041-6d034bd73caa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:35:52.013836 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013742    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-model-cache\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:52.013836 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013763    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-tmp-dir\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:52.013836 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.013784    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-home" (OuterVolumeSpecName: "home") pod "28339bb2-d7a4-48e3-8041-6d034bd73caa" (UID: "28339bb2-d7a4-48e3-8041-6d034bd73caa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:35:52.015373 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.015352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28339bb2-d7a4-48e3-8041-6d034bd73caa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28339bb2-d7a4-48e3-8041-6d034bd73caa" (UID: "28339bb2-d7a4-48e3-8041-6d034bd73caa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:35:52.015827 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.015809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-dshm" (OuterVolumeSpecName: "dshm") pod "28339bb2-d7a4-48e3-8041-6d034bd73caa" (UID: "28339bb2-d7a4-48e3-8041-6d034bd73caa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 22:35:52.015894 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.015837    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28339bb2-d7a4-48e3-8041-6d034bd73caa-kube-api-access-qwvwk" (OuterVolumeSpecName: "kube-api-access-qwvwk") pod "28339bb2-d7a4-48e3-8041-6d034bd73caa" (UID: "28339bb2-d7a4-48e3-8041-6d034bd73caa"). InnerVolumeSpecName "kube-api-access-qwvwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:35:52.115023 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.114986    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwvwk\" (UniqueName: \"kubernetes.io/projected/28339bb2-d7a4-48e3-8041-6d034bd73caa-kube-api-access-qwvwk\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:52.115023 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.115018    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-home\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:52.115023 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.115029    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28339bb2-d7a4-48e3-8041-6d034bd73caa-dshm\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:52.115252 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.115036    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28339bb2-d7a4-48e3-8041-6d034bd73caa-tls-certs\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:52.783122 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.783085    2575 generic.go:356] "Generic (PLEG): container finished" podID="28339bb2-d7a4-48e3-8041-6d034bd73caa" containerID="bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c" exitCode=0
Jul 23 22:35:52.783122 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.783124    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"
Jul 23 22:35:52.783625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.783165    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9" event={"ID":"28339bb2-d7a4-48e3-8041-6d034bd73caa","Type":"ContainerDied","Data":"bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c"}
Jul 23 22:35:52.783625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.783209    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9" event={"ID":"28339bb2-d7a4-48e3-8041-6d034bd73caa","Type":"ContainerDied","Data":"f266b051fa5f525834a80b206b190320b596dacd6115e91b46b82cadb89cae6b"}
Jul 23 22:35:52.783625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.783226    2575 scope.go:117] "RemoveContainer" containerID="bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c"
Jul 23 22:35:52.793058 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.793036    2575 scope.go:117] "RemoveContainer" containerID="bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c"
Jul 23 22:35:52.793346 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:35:52.793324    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c\": container with ID starting with bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c not found: ID does not exist" containerID="bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c"
Jul 23 22:35:52.793392 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.793363    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c"} err="failed to get container status \"bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c\": rpc error: code = NotFound desc = could not find container \"bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c\": container with ID starting with bcaaa0d66cf8b1d95328c549524c4942310a30dc0098369e8d253d62bd13270c not found: ID does not exist"
Jul 23 22:35:52.799669 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.799643    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"]
Jul 23 22:35:52.803614 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.803592    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6bf8de/rate-test-model-6bf8de-kserve-57588666dc-kjbf9"]
Jul 23 22:35:52.844375 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844339    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140371672127648"]
Jul 23 22:35:52.844774 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844759    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e57551b-c805-4768-ad73-327ea72ff12f" containerName="test-curl-7464-140371670992944"
Jul 23 22:35:52.844826 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844776    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e57551b-c805-4768-ad73-327ea72ff12f" containerName="test-curl-7464-140371670992944"
Jul 23 22:35:52.844826 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844792    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28339bb2-d7a4-48e3-8041-6d034bd73caa" containerName="main"
Jul 23 22:35:52.844826 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844797    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28339bb2-d7a4-48e3-8041-6d034bd73caa" containerName="main"
Jul 23 22:35:52.844925 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844865    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28339bb2-d7a4-48e3-8041-6d034bd73caa" containerName="main"
Jul 23 22:35:52.844925 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.844875    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e57551b-c805-4768-ad73-327ea72ff12f" containerName="test-curl-7464-140371670992944"
Jul 23 22:35:52.849529 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.849511    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:52.851929 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.851903    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rmhjm\""
Jul 23 22:35:52.853910 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.853887    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140371672127648"]
Jul 23 22:35:52.921360 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:52.921325    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrj7l\" (UniqueName: \"kubernetes.io/projected/ef026fab-b41f-492e-a357-c6466ca89a18-kube-api-access-lrj7l\") pod \"test-curl-7464-140371672127648\" (UID: \"ef026fab-b41f-492e-a357-c6466ca89a18\") " pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:53.022344 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.022302    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrj7l\" (UniqueName: \"kubernetes.io/projected/ef026fab-b41f-492e-a357-c6466ca89a18-kube-api-access-lrj7l\") pod \"test-curl-7464-140371672127648\" (UID: \"ef026fab-b41f-492e-a357-c6466ca89a18\") " pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:53.030705 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.030672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrj7l\" (UniqueName: \"kubernetes.io/projected/ef026fab-b41f-492e-a357-c6466ca89a18-kube-api-access-lrj7l\") pod \"test-curl-7464-140371672127648\" (UID: \"ef026fab-b41f-492e-a357-c6466ca89a18\") " pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:53.161501 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.161463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:53.306544 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.306518    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140371672127648"]
Jul 23 22:35:53.308497 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:35:53.308469    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef026fab_b41f_492e_a357_c6466ca89a18.slice/crio-cd485d0e42892699bac4b9f3fc95c47e2d955b24f69c06a3f28560fa7b845dec WatchSource:0}: Error finding container cd485d0e42892699bac4b9f3fc95c47e2d955b24f69c06a3f28560fa7b845dec: Status 404 returned error can't find the container with id cd485d0e42892699bac4b9f3fc95c47e2d955b24f69c06a3f28560fa7b845dec
Jul 23 22:35:53.790528 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.790498    2575 generic.go:356] "Generic (PLEG): container finished" podID="ef026fab-b41f-492e-a357-c6466ca89a18" containerID="6ebdd88bb18f17e5020fd1ce50d759e00adb19dfd36d2771295ec51749a323e6" exitCode=0
Jul 23 22:35:53.790958 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.790585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140371672127648" event={"ID":"ef026fab-b41f-492e-a357-c6466ca89a18","Type":"ContainerDied","Data":"6ebdd88bb18f17e5020fd1ce50d759e00adb19dfd36d2771295ec51749a323e6"}
Jul 23 22:35:53.790958 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:53.790650    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140371672127648" event={"ID":"ef026fab-b41f-492e-a357-c6466ca89a18","Type":"ContainerStarted","Data":"cd485d0e42892699bac4b9f3fc95c47e2d955b24f69c06a3f28560fa7b845dec"}
Jul 23 22:35:54.227718 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:54.227678    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28339bb2-d7a4-48e3-8041-6d034bd73caa" path="/var/lib/kubelet/pods/28339bb2-d7a4-48e3-8041-6d034bd73caa/volumes"
Jul 23 22:35:54.954602 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:54.954577    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:54.980161 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:54.980128    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140371672127648_ef026fab-b41f-492e-a357-c6466ca89a18/test-curl-7464-140371672127648/0.log"
Jul 23 22:35:55.009695 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.009668    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140371672127648"]
Jul 23 22:35:55.014792 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.014766    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140371672127648"]
Jul 23 22:35:55.040556 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.040531    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrj7l\" (UniqueName: \"kubernetes.io/projected/ef026fab-b41f-492e-a357-c6466ca89a18-kube-api-access-lrj7l\") pod \"ef026fab-b41f-492e-a357-c6466ca89a18\" (UID: \"ef026fab-b41f-492e-a357-c6466ca89a18\") "
Jul 23 22:35:55.042560 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.042535    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef026fab-b41f-492e-a357-c6466ca89a18-kube-api-access-lrj7l" (OuterVolumeSpecName: "kube-api-access-lrj7l") pod "ef026fab-b41f-492e-a357-c6466ca89a18" (UID: "ef026fab-b41f-492e-a357-c6466ca89a18"). InnerVolumeSpecName "kube-api-access-lrj7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:35:55.141913 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.141880    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrj7l\" (UniqueName: \"kubernetes.io/projected/ef026fab-b41f-492e-a357-c6466ca89a18-kube-api-access-lrj7l\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:55.208168 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.208125    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"]
Jul 23 22:35:55.208580 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.208566    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef026fab-b41f-492e-a357-c6466ca89a18" containerName="test-curl-7464-140371672127648"
Jul 23 22:35:55.208626 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.208582    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef026fab-b41f-492e-a357-c6466ca89a18" containerName="test-curl-7464-140371672127648"
Jul 23 22:35:55.208690 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.208678    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef026fab-b41f-492e-a357-c6466ca89a18" containerName="test-curl-7464-140371672127648"
Jul 23 22:35:55.219241 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.219201    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:55.220827 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.220792    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"]
Jul 23 22:35:55.221895 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.221872    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:35:55.343655 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.343601    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3208ee-5d9d-4363-bff1-7218c65647ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq\" (UID: \"1b3208ee-5d9d-4363-bff1-7218c65647ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:55.444348 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.444308    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3208ee-5d9d-4363-bff1-7218c65647ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq\" (UID: \"1b3208ee-5d9d-4363-bff1-7218c65647ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:55.444987 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.444966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3208ee-5d9d-4363-bff1-7218c65647ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq\" (UID: \"1b3208ee-5d9d-4363-bff1-7218c65647ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:55.531725 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.531689    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:55.670912 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.670867    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"]
Jul 23 22:35:55.672129 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:35:55.672091    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3208ee_5d9d_4363_bff1_7218c65647ac.slice/crio-b4b09b071de10a3567dd7a82ffde5b244c79fcb7bfe749ce72abc220c47594dd WatchSource:0}: Error finding container b4b09b071de10a3567dd7a82ffde5b244c79fcb7bfe749ce72abc220c47594dd: Status 404 returned error can't find the container with id b4b09b071de10a3567dd7a82ffde5b244c79fcb7bfe749ce72abc220c47594dd
Jul 23 22:35:55.805601 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.805566    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq" event={"ID":"1b3208ee-5d9d-4363-bff1-7218c65647ac","Type":"ContainerStarted","Data":"b4b09b071de10a3567dd7a82ffde5b244c79fcb7bfe749ce72abc220c47594dd"}
Jul 23 22:35:55.806843 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.806822    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:55.806976 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.806862    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd485d0e42892699bac4b9f3fc95c47e2d955b24f69c06a3f28560fa7b845dec"
Jul 23 22:35:55.817000 ip-10-0-143-223 kubenswrapper[2575]: E0723 22:35:55.816970    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-140371672127648\" is forbidden: User \"system:node:ip-10-0-143-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-223.ec2.internal' and this object" podUID="ef026fab-b41f-492e-a357-c6466ca89a18" pod="opendatahub/test-curl-7464-140371672127648"
Jul 23 22:35:55.866982 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.866920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:55.968919 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:55.968886    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:35:56.229357 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:56.229263    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef026fab-b41f-492e-a357-c6466ca89a18" path="/var/lib/kubelet/pods/ef026fab-b41f-492e-a357-c6466ca89a18/volumes"
Jul 23 22:35:56.814343 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:56.814309    2575 generic.go:356] "Generic (PLEG): container finished" podID="1b3208ee-5d9d-4363-bff1-7218c65647ac" containerID="5e2021c187460b50c51e82c67e8818d628faa2964a1596b127417585f6418857" exitCode=0
Jul 23 22:35:56.814516 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:56.814397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq" event={"ID":"1b3208ee-5d9d-4363-bff1-7218c65647ac","Type":"ContainerDied","Data":"5e2021c187460b50c51e82c67e8818d628faa2964a1596b127417585f6418857"}
Jul 23 22:35:57.963306 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:57.963281    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:58.066517 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:58.066476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3208ee-5d9d-4363-bff1-7218c65647ac-maas-api-service-ca\") pod \"1b3208ee-5d9d-4363-bff1-7218c65647ac\" (UID: \"1b3208ee-5d9d-4363-bff1-7218c65647ac\") "
Jul 23 22:35:58.066932 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:58.066908    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b3208ee-5d9d-4363-bff1-7218c65647ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b3208ee-5d9d-4363-bff1-7218c65647ac" (UID: "1b3208ee-5d9d-4363-bff1-7218c65647ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:35:58.168020 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:58.167986    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3208ee-5d9d-4363-bff1-7218c65647ac-maas-api-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:35:58.827583 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:58.827550    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"
Jul 23 22:35:58.827583 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:58.827571    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq" event={"ID":"1b3208ee-5d9d-4363-bff1-7218c65647ac","Type":"ContainerDied","Data":"b4b09b071de10a3567dd7a82ffde5b244c79fcb7bfe749ce72abc220c47594dd"}
Jul 23 22:35:58.827828 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:35:58.827616    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4b09b071de10a3567dd7a82ffde5b244c79fcb7bfe749ce72abc220c47594dd"
Jul 23 22:36:05.774910 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:05.774862    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:05.780702 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:05.780668    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"]
Jul 23 22:36:05.785047 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:05.785021    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1c273d-29747430-jr6gj"]
Jul 23 22:36:06.227299 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:06.227262    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3643d543-8857-4890-9412-999f33812dd0" path="/var/lib/kubelet/pods/3643d543-8857-4890-9412-999f33812dd0/volumes"
Jul 23 22:36:10.304361 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:10.304327    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:10.317817 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:10.317786    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"]
Jul 23 22:36:10.320121 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:10.320099    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1c273d-4zmfq"]
Jul 23 22:36:10.694398 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:10.694364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:12.227306 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:12.227267    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b3208ee-5d9d-4363-bff1-7218c65647ac" path="/var/lib/kubelet/pods/1b3208ee-5d9d-4363-bff1-7218c65647ac/volumes"
Jul 23 22:36:16.699805 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:16.699766    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:16.798780 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:16.798745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:16.998466 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:16.998379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:17.221789 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.221753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:17.376882 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.376846    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"]
Jul 23 22:36:17.377253 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.377240    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3208ee-5d9d-4363-bff1-7218c65647ac" containerName="revoke-keys"
Jul 23 22:36:17.377313 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.377255    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3208ee-5d9d-4363-bff1-7218c65647ac" containerName="revoke-keys"
Jul 23 22:36:17.377350 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.377335    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3208ee-5d9d-4363-bff1-7218c65647ac" containerName="revoke-keys"
Jul 23 22:36:17.382257 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.382240    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:17.386033 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.386010    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7xtsk\""
Jul 23 22:36:17.399149 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.399119    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"]
Jul 23 22:36:17.544485 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.544445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aede8f7-1f3c-409f-aa5d-53c691830489-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms\" (UID: \"4aede8f7-1f3c-409f-aa5d-53c691830489\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:17.645759 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.645658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aede8f7-1f3c-409f-aa5d-53c691830489-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms\" (UID: \"4aede8f7-1f3c-409f-aa5d-53c691830489\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:17.646357 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.646330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aede8f7-1f3c-409f-aa5d-53c691830489-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms\" (UID: \"4aede8f7-1f3c-409f-aa5d-53c691830489\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:17.692714 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.692675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:17.836856 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.836826    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"]
Jul 23 22:36:17.839267 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:36:17.839236    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aede8f7_1f3c_409f_aa5d_53c691830489.slice/crio-487b1a4d0a45ca659504e0f3e72d3ddc058df19338f8cdf4de4b1ee3158f477a WatchSource:0}: Error finding container 487b1a4d0a45ca659504e0f3e72d3ddc058df19338f8cdf4de4b1ee3158f477a: Status 404 returned error can't find the container with id 487b1a4d0a45ca659504e0f3e72d3ddc058df19338f8cdf4de4b1ee3158f477a
Jul 23 22:36:17.948372 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.948284    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms" event={"ID":"4aede8f7-1f3c-409f-aa5d-53c691830489","Type":"ContainerStarted","Data":"487b1a4d0a45ca659504e0f3e72d3ddc058df19338f8cdf4de4b1ee3158f477a"}
Jul 23 22:36:17.951338 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:17.951311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:18.052518 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:18.052482    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:18.515342 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:18.515312    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:36:18.522417 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:18.522391    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:36:18.955069 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:18.955025    2575 generic.go:356] "Generic (PLEG): container finished" podID="4aede8f7-1f3c-409f-aa5d-53c691830489" containerID="af8a428488c73f27314035acc73288de94b5f00a569523284e28dbfdd2c20726" exitCode=0
Jul 23 22:36:18.955474 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:18.955134    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms" event={"ID":"4aede8f7-1f3c-409f-aa5d-53c691830489","Type":"ContainerDied","Data":"af8a428488c73f27314035acc73288de94b5f00a569523284e28dbfdd2c20726"}
Jul 23 22:36:19.012360 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:19.012329    2575 scope.go:117] "RemoveContainer" containerID="36afe9566945df43f6bd7c0e114ff2b0d9970ef11e3187617ce43907efd86ecd"
Jul 23 22:36:20.101778 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.101749    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:20.270334 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.270234    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aede8f7-1f3c-409f-aa5d-53c691830489-maas-api-service-ca\") pod \"4aede8f7-1f3c-409f-aa5d-53c691830489\" (UID: \"4aede8f7-1f3c-409f-aa5d-53c691830489\") "
Jul 23 22:36:20.270579 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.270554    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aede8f7-1f3c-409f-aa5d-53c691830489-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4aede8f7-1f3c-409f-aa5d-53c691830489" (UID: "4aede8f7-1f3c-409f-aa5d-53c691830489"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:36:20.371268 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.371228    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aede8f7-1f3c-409f-aa5d-53c691830489-maas-api-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:36:20.968365 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.968337    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"
Jul 23 22:36:20.968530 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.968335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms" event={"ID":"4aede8f7-1f3c-409f-aa5d-53c691830489","Type":"ContainerDied","Data":"487b1a4d0a45ca659504e0f3e72d3ddc058df19338f8cdf4de4b1ee3158f477a"}
Jul 23 22:36:20.968530 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:20.968468    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="487b1a4d0a45ca659504e0f3e72d3ddc058df19338f8cdf4de4b1ee3158f477a"
Jul 23 22:36:27.791950 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:27.791908    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"]
Jul 23 22:36:27.792398 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:27.792190    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" podUID="a5d8b969-e544-47ae-b914-01f1d945af42" containerName="maas-api" containerID="cri-o://ea2489c451552d40f00ee283989ae0c958b3944332d3b9825fc87caeb09567b4" gracePeriod=30
Jul 23 22:36:27.818074 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:27.818044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:27.819961 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:27.819932    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"]
Jul 23 22:36:27.822552 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:27.822531    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6bf8de-29747430-r5l5v"]
Jul 23 22:36:28.011849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.011814    2575 generic.go:356] "Generic (PLEG): container finished" podID="a5d8b969-e544-47ae-b914-01f1d945af42" containerID="ea2489c451552d40f00ee283989ae0c958b3944332d3b9825fc87caeb09567b4" exitCode=0
Jul 23 22:36:28.012019 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.011890    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" event={"ID":"a5d8b969-e544-47ae-b914-01f1d945af42","Type":"ContainerDied","Data":"ea2489c451552d40f00ee283989ae0c958b3944332d3b9825fc87caeb09567b4"}
Jul 23 22:36:28.055501 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.055434    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:36:28.139871 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.139837    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5bbj\" (UniqueName: \"kubernetes.io/projected/a5d8b969-e544-47ae-b914-01f1d945af42-kube-api-access-g5bbj\") pod \"a5d8b969-e544-47ae-b914-01f1d945af42\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") "
Jul 23 22:36:28.139871 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.139870    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls\") pod \"a5d8b969-e544-47ae-b914-01f1d945af42\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") "
Jul 23 22:36:28.140121 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.139911    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5d8b969-e544-47ae-b914-01f1d945af42-openshift-service-ca\") pod \"a5d8b969-e544-47ae-b914-01f1d945af42\" (UID: \"a5d8b969-e544-47ae-b914-01f1d945af42\") "
Jul 23 22:36:28.140316 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.140291    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5d8b969-e544-47ae-b914-01f1d945af42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5d8b969-e544-47ae-b914-01f1d945af42" (UID: "a5d8b969-e544-47ae-b914-01f1d945af42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 22:36:28.142007 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.141984    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d8b969-e544-47ae-b914-01f1d945af42-kube-api-access-g5bbj" (OuterVolumeSpecName: "kube-api-access-g5bbj") pod "a5d8b969-e544-47ae-b914-01f1d945af42" (UID: "a5d8b969-e544-47ae-b914-01f1d945af42"). InnerVolumeSpecName "kube-api-access-g5bbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 22:36:28.142098 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.142059    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5d8b969-e544-47ae-b914-01f1d945af42" (UID: "a5d8b969-e544-47ae-b914-01f1d945af42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 22:36:28.229150 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.229113    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc5c474-d5bd-4af1-b59a-c1b4923dbae9" path="/var/lib/kubelet/pods/6bc5c474-d5bd-4af1-b59a-c1b4923dbae9/volumes"
Jul 23 22:36:28.241091 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.241062    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5bbj\" (UniqueName: \"kubernetes.io/projected/a5d8b969-e544-47ae-b914-01f1d945af42-kube-api-access-g5bbj\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:36:28.241216 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.241095    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5d8b969-e544-47ae-b914-01f1d945af42-maas-api-tls\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:36:28.241216 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:28.241110    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5d8b969-e544-47ae-b914-01f1d945af42-openshift-service-ca\") on node \"ip-10-0-143-223.ec2.internal\" DevicePath \"\""
Jul 23 22:36:29.019179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:29.019144    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm" event={"ID":"a5d8b969-e544-47ae-b914-01f1d945af42","Type":"ContainerDied","Data":"61f81bdedd26f02f5599a1d394497d5afe045c25dbe182820e0035cf67ef6ea0"}
Jul 23 22:36:29.019179 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:29.019172    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"
Jul 23 22:36:29.019729 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:29.019203    2575 scope.go:117] "RemoveContainer" containerID="ea2489c451552d40f00ee283989ae0c958b3944332d3b9825fc87caeb09567b4"
Jul 23 22:36:29.035165 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:29.035137    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"]
Jul 23 22:36:29.038364 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:29.038338    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6bf8de-779875d8f7-9l5zm"]
Jul 23 22:36:30.227941 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:30.227897    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5d8b969-e544-47ae-b914-01f1d945af42" path="/var/lib/kubelet/pods/a5d8b969-e544-47ae-b914-01f1d945af42/volumes"
Jul 23 22:36:32.453655 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:32.447908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:32.454263 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:32.454227    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"]
Jul 23 22:36:32.456210 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:32.456179    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6bf8de-ftrms"]
Jul 23 22:36:32.857423 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:32.857368    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:34.227657 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:34.227596    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aede8f7-1f3c-409f-aa5d-53c691830489" path="/var/lib/kubelet/pods/4aede8f7-1f3c-409f-aa5d-53c691830489/volumes"
Jul 23 22:36:38.848507 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:38.848473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:38.949460 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:38.949422    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:39.134895 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:39.134816    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:39.434912 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:39.434822    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:45.512877 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:45.512848    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gwc67_e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a/manager/0.log"
Jul 23 22:36:45.636718 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:45.636683    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xrm96_1f66ef48-03cc-434d-bd74-8f753afe23a9/manager/0.log"
Jul 23 22:36:45.876464 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:45.876437    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jkw4d_19dfbc68-20e8-41d0-809f-a35b8939ee66/server/0.log"
Jul 23 22:36:46.112726 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:46.112690    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7bc97c8c-7blc5_b37abf6a-9a62-44ff-b814-f9ed3096f617/copy-manifests/0.log"
Jul 23 22:36:46.119845 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:46.119818    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7bc97c8c-7blc5_b37abf6a-9a62-44ff-b814-f9ed3096f617/manager/0.log"
Jul 23 22:36:47.019920 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:47.019888    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747430-8nww2_51238466-af83-4830-bbdd-f99b964e6cd9/cleanup/0.log"
Jul 23 22:36:47.141577 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:47.141547    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-pl4k2_bf3375f2-cce1-4df0-8e1c-94f7e11115bd/postgres/0.log"
Jul 23 22:36:47.975677 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:47.975621    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp_40abff84-09dc-4298-93d2-71f43689ac98/util/0.log"
Jul 23 22:36:47.982245 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:47.982220    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp_40abff84-09dc-4298-93d2-71f43689ac98/pull/0.log"
Jul 23 22:36:47.989867 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:47.989843    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp_40abff84-09dc-4298-93d2-71f43689ac98/extract/0.log"
Jul 23 22:36:48.124368 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.124337    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6_a1426ae8-168e-4f18-b921-23138b673f0f/extract/0.log"
Jul 23 22:36:48.130191 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.130168    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6_a1426ae8-168e-4f18-b921-23138b673f0f/util/0.log"
Jul 23 22:36:48.136094 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.136075    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6_a1426ae8-168e-4f18-b921-23138b673f0f/pull/0.log"
Jul 23 22:36:48.260079 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.259996    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr_41159bfa-3f68-40c5-b818-34c1c484293d/util/0.log"
Jul 23 22:36:48.266472 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.266449    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr_41159bfa-3f68-40c5-b818-34c1c484293d/pull/0.log"
Jul 23 22:36:48.273337 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.273308    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr_41159bfa-3f68-40c5-b818-34c1c484293d/extract/0.log"
Jul 23 22:36:48.396045 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.396008    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7_a3acd915-4bc2-4815-8d0d-44d81fa2f418/util/0.log"
Jul 23 22:36:48.402930 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.402898    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7_a3acd915-4bc2-4815-8d0d-44d81fa2f418/pull/0.log"
Jul 23 22:36:48.409097 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.409081    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7_a3acd915-4bc2-4815-8d0d-44d81fa2f418/extract/0.log"
Jul 23 22:36:48.667177 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.667148    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z9k6h_810defa1-272c-48eb-85a4-5e6138be1e34/manager/0.log"
Jul 23 22:36:48.782989 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.782960    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qz828_779ad542-316e-4ba6-a1a9-4581bf9a4c99/manager/0.log"
Jul 23 22:36:48.900309 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:48.900280    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2ncs6_bfd8d6b4-d360-43d4-b11f-793a9d63b9a3/kuadrant-console-plugin/0.log"
Jul 23 22:36:49.019005 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:49.018928    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-885tg_74c91f55-0b19-4063-ae89-630510eed8ff/registry-server/0.log"
Jul 23 22:36:49.144070 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:49.144040    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wplkg_9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724/manager/0.log"
Jul 23 22:36:49.392794 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:49.392760    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2ncd7_8bca412e-f8b7-40bc-bd2b-2343f4da7bbf/manager/0.log"
Jul 23 22:36:50.010277 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:50.010244    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc6f65d97-79nl4_aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998/kube-auth-proxy/0.log"
Jul 23 22:36:50.500025 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:50.499995    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-rspp6_5f5e8d22-53ca-4590-8af0-ccc1f1dfb503/payload-processing/0.log"
Jul 23 22:36:50.623043 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:50.623010    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9b7f69fb-bxs2f_cdcef278-0391-4f9d-b80c-7f404fb55655/router/0.log"
Jul 23 22:36:51.000941 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:51.000910    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-t5m9t_93dc1d7d-9e6f-46b5-901f-1afe5e94d0ea/main/0.log"
Jul 23 22:36:51.245174 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:51.245146    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-7nzlj_e81a6a8d-8e9f-47ca-90ff-000143990135/main/0.log"
Jul 23 22:36:51.624378 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:51.624340    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-v9ql8_16d63835-b27b-4fbc-9714-ffa68e7568ac/main/0.log"
Jul 23 22:36:57.953984 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.953949    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wczc/must-gather-vkv8c"]
Jul 23 22:36:57.954435 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.954345    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aede8f7-1f3c-409f-aa5d-53c691830489" containerName="revoke-keys"
Jul 23 22:36:57.954435 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.954358    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aede8f7-1f3c-409f-aa5d-53c691830489" containerName="revoke-keys"
Jul 23 22:36:57.954435 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.954376    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5d8b969-e544-47ae-b914-01f1d945af42" containerName="maas-api"
Jul 23 22:36:57.954435 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.954381    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5d8b969-e544-47ae-b914-01f1d945af42" containerName="maas-api"
Jul 23 22:36:57.954567 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.954449    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5d8b969-e544-47ae-b914-01f1d945af42" containerName="maas-api"
Jul 23 22:36:57.954567 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.954459    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aede8f7-1f3c-409f-aa5d-53c691830489" containerName="revoke-keys"
Jul 23 22:36:57.957995 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.957973    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:57.960480 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.960452    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wczc\"/\"kube-root-ca.crt\""
Jul 23 22:36:57.960599 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.960524    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wczc\"/\"openshift-service-ca.crt\""
Jul 23 22:36:57.961254 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.961234    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4wczc\"/\"default-dockercfg-bxs7g\""
Jul 23 22:36:57.967524 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.967500    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wczc/must-gather-vkv8c"]
Jul 23 22:36:57.981818 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:57.981767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:58.087038 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.086992    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log" limit=1024
Jul 23 22:36:58.104447 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.104409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99zfp\" (UniqueName: \"kubernetes.io/projected/593e4063-441b-4dbb-8eb5-cd9416156dd6-kube-api-access-99zfp\") pod \"must-gather-vkv8c\" (UID: \"593e4063-441b-4dbb-8eb5-cd9416156dd6\") " pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.104601 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.104522    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/593e4063-441b-4dbb-8eb5-cd9416156dd6-must-gather-output\") pod \"must-gather-vkv8c\" (UID: \"593e4063-441b-4dbb-8eb5-cd9416156dd6\") " pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.205769 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.205675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/593e4063-441b-4dbb-8eb5-cd9416156dd6-must-gather-output\") pod \"must-gather-vkv8c\" (UID: \"593e4063-441b-4dbb-8eb5-cd9416156dd6\") " pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.205769 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.205730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99zfp\" (UniqueName: \"kubernetes.io/projected/593e4063-441b-4dbb-8eb5-cd9416156dd6-kube-api-access-99zfp\") pod \"must-gather-vkv8c\" (UID: \"593e4063-441b-4dbb-8eb5-cd9416156dd6\") " pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.206054 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.206031    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/593e4063-441b-4dbb-8eb5-cd9416156dd6-must-gather-output\") pod \"must-gather-vkv8c\" (UID: \"593e4063-441b-4dbb-8eb5-cd9416156dd6\") " pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.213349 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.213324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99zfp\" (UniqueName: \"kubernetes.io/projected/593e4063-441b-4dbb-8eb5-cd9416156dd6-kube-api-access-99zfp\") pod \"must-gather-vkv8c\" (UID: \"593e4063-441b-4dbb-8eb5-cd9416156dd6\") " pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.269668 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.269603    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wczc/must-gather-vkv8c"
Jul 23 22:36:58.411869 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:58.411833    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wczc/must-gather-vkv8c"]
Jul 23 22:36:58.413440 ip-10-0-143-223 kubenswrapper[2575]: W0723 22:36:58.413411    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod593e4063_441b_4dbb_8eb5_cd9416156dd6.slice/crio-fa685cfd568364a6d6246dcfcaa16264e020231784d7856c5f501ce5fe7d78af WatchSource:0}: Error finding container fa685cfd568364a6d6246dcfcaa16264e020231784d7856c5f501ce5fe7d78af: Status 404 returned error can't find the container with id fa685cfd568364a6d6246dcfcaa16264e020231784d7856c5f501ce5fe7d78af
Jul 23 22:36:59.202064 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:36:59.202023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wczc/must-gather-vkv8c" event={"ID":"593e4063-441b-4dbb-8eb5-cd9416156dd6","Type":"ContainerStarted","Data":"fa685cfd568364a6d6246dcfcaa16264e020231784d7856c5f501ce5fe7d78af"}
Jul 23 22:37:00.211046 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:00.211002    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wczc/must-gather-vkv8c" event={"ID":"593e4063-441b-4dbb-8eb5-cd9416156dd6","Type":"ContainerStarted","Data":"b7b487a13b92694fae78fc785762a1d69842f8c9f9662bff74c9c62d19fa2276"}
Jul 23 22:37:00.211527 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:00.211059    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wczc/must-gather-vkv8c" event={"ID":"593e4063-441b-4dbb-8eb5-cd9416156dd6","Type":"ContainerStarted","Data":"76b6ee4f74ae82b2484a0ad6308b533d98f65484ea24d82697213413a631b63d"}
Jul 23 22:37:00.228176 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:00.228121    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wczc/must-gather-vkv8c" podStartSLOduration=2.368177012 podStartE2EDuration="3.228104517s" totalImagesPullingTime="859.927505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:37:00.227031856 +0000 UTC m=+2742.488031604" watchObservedRunningTime="2026-07-23 22:37:00.228104517 +0000 UTC m=+2742.489104244"
Jul 23 22:37:00.851329 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:00.851281    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-29qg9_886ede06-fccf-4379-831a-41c244ab7387/global-pull-secret-syncer/0.log"
Jul 23 22:37:00.986577 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:00.986543    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mfvsh_a91e5420-94f1-4941-82f5-1cf1eae0496c/konnectivity-agent/0.log"
Jul 23 22:37:01.137265 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:01.137190    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-223.ec2.internal_aa48ab49f2c0e7a214bbd553b8e1864c/haproxy/0.log"
Jul 23 22:37:06.028923 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.028864    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp_40abff84-09dc-4298-93d2-71f43689ac98/extract/0.log"
Jul 23 22:37:06.050851 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.050816    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp_40abff84-09dc-4298-93d2-71f43689ac98/util/0.log"
Jul 23 22:37:06.073690 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.073576    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p24qp_40abff84-09dc-4298-93d2-71f43689ac98/pull/0.log"
Jul 23 22:37:06.113134 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.113106    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6_a1426ae8-168e-4f18-b921-23138b673f0f/extract/0.log"
Jul 23 22:37:06.143292 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.143249    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6_a1426ae8-168e-4f18-b921-23138b673f0f/util/0.log"
Jul 23 22:37:06.165592 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.165555    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e07qwg6_a1426ae8-168e-4f18-b921-23138b673f0f/pull/0.log"
Jul 23 22:37:06.194387 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.194288    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr_41159bfa-3f68-40c5-b818-34c1c484293d/extract/0.log"
Jul 23 22:37:06.215861 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.215768    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr_41159bfa-3f68-40c5-b818-34c1c484293d/util/0.log"
Jul 23 22:37:06.241363 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.241335    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tssdr_41159bfa-3f68-40c5-b818-34c1c484293d/pull/0.log"
Jul 23 22:37:06.267625 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.267592    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7_a3acd915-4bc2-4815-8d0d-44d81fa2f418/extract/0.log"
Jul 23 22:37:06.294901 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.294822    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7_a3acd915-4bc2-4815-8d0d-44d81fa2f418/util/0.log"
Jul 23 22:37:06.315373 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.315338    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef197ln7_a3acd915-4bc2-4815-8d0d-44d81fa2f418/pull/0.log"
Jul 23 22:37:06.668467 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.668429    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z9k6h_810defa1-272c-48eb-85a4-5e6138be1e34/manager/0.log"
Jul 23 22:37:06.711219 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.711183    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qz828_779ad542-316e-4ba6-a1a9-4581bf9a4c99/manager/0.log"
Jul 23 22:37:06.740852 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.740826    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2ncs6_bfd8d6b4-d360-43d4-b11f-793a9d63b9a3/kuadrant-console-plugin/0.log"
Jul 23 22:37:06.791393 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.791358    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-885tg_74c91f55-0b19-4063-ae89-630510eed8ff/registry-server/0.log"
Jul 23 22:37:06.972970 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:06.972887    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wplkg_9f0ba7bc-4ee0-403f-b2d9-6e92dd20d724/manager/0.log"
Jul 23 22:37:07.100436 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:07.100397    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2ncd7_8bca412e-f8b7-40bc-bd2b-2343f4da7bbf/manager/0.log"
Jul 23 22:37:08.971438 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:08.971400    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-n9zrt_335b3ca4-7b4a-4e04-b53f-f24a0c7b7e58/cluster-monitoring-operator/0.log"
Jul 23 22:37:09.225149 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:09.224939    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cb9xj_f56cb8fc-c371-4fc0-998a-843ff6237672/node-exporter/0.log"
Jul 23 22:37:09.249811 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:09.249780    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cb9xj_f56cb8fc-c371-4fc0-998a-843ff6237672/kube-rbac-proxy/0.log"
Jul 23 22:37:09.277145 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:09.277113    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cb9xj_f56cb8fc-c371-4fc0-998a-843ff6237672/init-textfile/0.log"
Jul 23 22:37:09.776845 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:09.776813    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-s7nc6_62a1ff30-1fbe-4e62-8287-a42b865c42a6/prometheus-operator-admission-webhook/0.log"
Jul 23 22:37:10.705187 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.705142    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"]
Jul 23 22:37:10.711172 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.711145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.719211 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.719154    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"]
Jul 23 22:37:10.744021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.743554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-proc\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.744021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.743677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-podres\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.744021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.743754    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffb78\" (UniqueName: \"kubernetes.io/projected/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-kube-api-access-ffb78\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.744021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.743843    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-sys\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.744021 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.743889    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-lib-modules\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.844655 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-sys\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.844849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-lib-modules\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.844849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844727    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-sys\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.844849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-proc\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.844849 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-podres\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.845076 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844865    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-proc\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.845076 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-lib-modules\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.845076 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffb78\" (UniqueName: \"kubernetes.io/projected/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-kube-api-access-ffb78\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.845076 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.844961    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-podres\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:10.854269 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:10.854200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffb78\" (UniqueName: \"kubernetes.io/projected/0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5-kube-api-access-ffb78\") pod \"perf-node-gather-daemonset-vn28k\" (UID: \"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5\") " pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:11.034750 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:11.034660    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:11.231651 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:11.231591    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"]
Jul 23 22:37:11.342872 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:11.342776    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k" event={"ID":"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5","Type":"ContainerStarted","Data":"54550f1ece210d4bff0326233677a1626ff89af9f67ef9680bc7d3910ba1e6ab"}
Jul 23 22:37:11.740391 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:11.740357    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/1.log"
Jul 23 22:37:11.747982 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:11.747954    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s9bnx_9d931f74-1cd1-40fb-ba18-c613477a3784/console-operator/2.log"
Jul 23 22:37:12.223651 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:12.223601    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6986648f48-f86gf_0da43441-b5b6-4837-9adf-4a6407d7c859/console/0.log"
Jul 23 22:37:12.259017 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:12.258971    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-rcg2x_a7c2c9d0-7520-49e3-a45f-2f9a84d08e99/download-server/0.log"
Jul 23 22:37:12.354239 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:12.354190    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k" event={"ID":"0c61ec8d-6aa9-4b4e-b4a9-06f4b30340c5","Type":"ContainerStarted","Data":"b13f3729dd89712cfb329c4e34c289b6c2eda581baef5627138b66fa30930fc5"}
Jul 23 22:37:12.354564 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:12.354280    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:12.371304 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:12.371245    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k" podStartSLOduration=2.371225778 podStartE2EDuration="2.371225778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:37:12.36928058 +0000 UTC m=+2754.630280342" watchObservedRunningTime="2026-07-23 22:37:12.371225778 +0000 UTC m=+2754.632225507"
Jul 23 22:37:12.812269 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:12.812240    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-67d5s_09c67a01-6139-4677-b7b1-0cf92777ccfb/volume-data-source-validator/0.log"
Jul 23 22:37:13.864700 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:13.864672    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v6d4c_27d52fef-85f6-496c-b787-1f334f4c3f33/dns/0.log"
Jul 23 22:37:13.885208 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:13.885183    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v6d4c_27d52fef-85f6-496c-b787-1f334f4c3f33/kube-rbac-proxy/0.log"
Jul 23 22:37:13.993979 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:13.993947    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sftqj_d8b2d5df-b34f-4985-b490-555ffb020fcc/dns-node-resolver/0.log"
Jul 23 22:37:14.603642 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:14.603598    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz8gs_b2d08fc9-7fee-4c81-9b23-ffad61e60a7b/node-ca/0.log"
Jul 23 22:37:15.763462 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:15.763428    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc6f65d97-79nl4_aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998/kube-auth-proxy/0.log"
Jul 23 22:37:15.935674 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:15.935642    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-rspp6_5f5e8d22-53ca-4590-8af0-ccc1f1dfb503/payload-processing/0.log"
Jul 23 22:37:15.962866 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:15.962836    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9b7f69fb-bxs2f_cdcef278-0391-4f9d-b80c-7f404fb55655/router/0.log"
Jul 23 22:37:16.628331 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:16.628296    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hjj5g_ddf53267-738e-4f2f-852b-9ea39603709c/serve-healthcheck-canary/0.log"
Jul 23 22:37:17.144520 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:17.144491    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-4k9jn_ea1e6f27-49dc-457b-871f-699f016f9bdf/insights-operator/0.log"
Jul 23 22:37:17.146578 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:17.146553    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-4k9jn_ea1e6f27-49dc-457b-871f-699f016f9bdf/insights-operator/1.log"
Jul 23 22:37:17.168265 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:17.168231    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d4tk2_9d885894-adcb-4801-aabc-8618d6acb004/kube-rbac-proxy/0.log"
Jul 23 22:37:17.189503 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:17.189474    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d4tk2_9d885894-adcb-4801-aabc-8618d6acb004/exporter/0.log"
Jul 23 22:37:17.210724 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:17.210693    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d4tk2_9d885894-adcb-4801-aabc-8618d6acb004/extractor/0.log"
Jul 23 22:37:18.374173 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:18.374139    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4wczc/perf-node-gather-daemonset-vn28k"
Jul 23 22:37:19.057116 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:19.057070    2575 scope.go:117] "RemoveContainer" containerID="5cb8e5505ee18c0363dfe040a05e053361c0c1b768b3fd64b80cab189f952324"
Jul 23 22:37:19.621855 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:19.621817    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gwc67_e7b9b7a0-ffd3-4b4d-bf26-ad29ee220f9a/manager/0.log"
Jul 23 22:37:19.882773 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:19.882593    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xrm96_1f66ef48-03cc-434d-bd74-8f753afe23a9/manager/0.log"
Jul 23 22:37:20.022146 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:20.022107    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jkw4d_19dfbc68-20e8-41d0-809f-a35b8939ee66/server/0.log"
Jul 23 22:37:20.120684 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:20.120573    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7bc97c8c-7blc5_b37abf6a-9a62-44ff-b814-f9ed3096f617/manager/0.log"
Jul 23 22:37:20.142611 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:20.142527    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7bc97c8c-7blc5_b37abf6a-9a62-44ff-b814-f9ed3096f617/copy-manifests/0.log"
Jul 23 22:37:22.231424 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:22.231320    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc6f65d97-79nl4_aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998/kube-auth-proxy/0.log"
Jul 23 22:37:22.370936 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:22.370907    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-rspp6_5f5e8d22-53ca-4590-8af0-ccc1f1dfb503/payload-processing/0.log"
Jul 23 22:37:22.383351 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:22.383324    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9b7f69fb-bxs2f_cdcef278-0391-4f9d-b80c-7f404fb55655/router/0.log"
Jul 23 22:37:23.761562 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:23.761533    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc6f65d97-79nl4_aaef0f2b-3df5-4d88-bedc-ca4a6f6ae998/kube-auth-proxy/0.log"
Jul 23 22:37:23.900047 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:23.900002    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-rspp6_5f5e8d22-53ca-4590-8af0-ccc1f1dfb503/payload-processing/0.log"
Jul 23 22:37:23.916520 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:23.916489    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f9b7f69fb-bxs2f_cdcef278-0391-4f9d-b80c-7f404fb55655/router/0.log"
Jul 23 22:37:26.581920 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:26.581884    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-vs5vk_e46ed525-9024-4013-9810-e6c7972fb437/migrator/0.log"
Jul 23 22:37:26.601363 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:26.601324    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-vs5vk_e46ed525-9024-4013-9810-e6c7972fb437/graceful-termination/0.log"
Jul 23 22:37:26.955106 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:26.955063    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-95fgg_b22450ac-7815-41e8-954d-d065f8a75de5/kube-storage-version-migrator-operator/1.log"
Jul 23 22:37:26.955925 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:26.955905    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-95fgg_b22450ac-7815-41e8-954d-d065f8a75de5/kube-storage-version-migrator-operator/0.log"
Jul 23 22:37:27.934870 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:27.934790    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8bn87_49b50422-b290-4646-babf-e85f78ea429c/kube-multus/0.log"
Jul 23 22:37:28.265280 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.265181    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/kube-multus-additional-cni-plugins/0.log"
Jul 23 22:37:28.285433 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.285402    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/egress-router-binary-copy/0.log"
Jul 23 22:37:28.306090 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.306060    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/cni-plugins/0.log"
Jul 23 22:37:28.326487 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.326459    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/bond-cni-plugin/0.log"
Jul 23 22:37:28.346402 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.346371    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/routeoverride-cni/0.log"
Jul 23 22:37:28.367827 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.367767    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/whereabouts-cni-bincopy/0.log"
Jul 23 22:37:28.388289 ip-10-0-143-223 kubenswrapper[2575]: I0723 22:37:28.388261    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vc8qb_5209be5a-03c5-4cf0-b6bc-dbf4acf85df3/whereabouts-cni/0.log"