Jul 07 12:40:23.620613 ip-10-0-132-141 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 12:40:23.620627 ip-10-0-132-141 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 12:40:23.620637 ip-10-0-132-141 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 12:40:23.620956 ip-10-0-132-141 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 12:40:33.768651 ip-10-0-132-141 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 12:40:33.768668 ip-10-0-132-141 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a81ec39e9a2a4bd78b4550ebb308df8e --
Jul 07 12:42:10.853018 ip-10-0-132-141 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 12:42:11.245537 ip-10-0-132-141 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 12:42:11.245537 ip-10-0-132-141 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 12:42:11.245537 ip-10-0-132-141 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 12:42:11.245537 ip-10-0-132-141 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 12:42:11.245537 ip-10-0-132-141 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 12:42:11.248386 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.248295    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 12:42:11.253675 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253655    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 12:42:11.253675 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253672    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 12:42:11.253675 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253676    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 12:42:11.253675 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253680    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253685    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253689    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253693    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253696    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253699    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253703    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253705    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253708    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253711    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253713    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253716    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253718    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253721    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253740    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253743    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253746    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253749    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253751    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253755    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 12:42:11.253847 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253766    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253769    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253772    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253775    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253777    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253780    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253782    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253786    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253789    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253791    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253794    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253797    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253800    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253802    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253805    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253808    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253810    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253813    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253816    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 12:42:11.254319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253818    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253822    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253825    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253828    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253831    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253833    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253836    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253838    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253846    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253848    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253851    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253855    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253858    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253860    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253863    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253865    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253868    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253870    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253873    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 12:42:11.254807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253875    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253878    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253880    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253883    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253885    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253888    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253890    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253893    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253897    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253900    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253902    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253905    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253908    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253910    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253913    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253915    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253918    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253920    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253923    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253926    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 12:42:11.255278 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253928    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253931    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253934    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253937    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253939    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253942    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253944    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253947    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253949    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253952    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253954    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253957    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253959    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253962    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253964    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253967    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253969    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.253972    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254639    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254645    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 12:42:11.255837 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254650    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254652    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254655    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254658    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254660    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254663    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254665    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254668    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254671    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254673    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254676    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254678    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254681    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254684    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254686    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254689    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254692    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254694    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254697    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 12:42:11.256321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254701    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254705    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254708    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254712    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254716    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254720    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254738    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254741    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254744    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254746    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254749    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254752    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254755    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254758    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254761    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254764    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254767    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254769    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254772    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 12:42:11.256809 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254774    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254777    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254779    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254783    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254785    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254788    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254790    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254792    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254795    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254798    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254801    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254804    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254806    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254809    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254811    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254814    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254816    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254819    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254822    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254824    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 12:42:11.257277 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254826    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254829    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254831    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254834    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254837    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254839    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254843    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254845    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254848    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254851    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254853    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254856    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254858    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254861    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254864    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254866    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254869    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254871    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254874    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 12:42:11.257776 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254876    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254879    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254881    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254884    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254888    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254890    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254892    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254895    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254898    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254900    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254903    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254905    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254908    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254911    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254913    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254915    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254918    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254921    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254923    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.254926    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 12:42:11.258236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255293    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255299    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255301    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255304    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255309    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255313    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255316    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255319    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255322    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255325    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255328    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255331    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255333    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255336    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255339    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255342    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255345    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255348    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255350    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 12:42:11.258706 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255352    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255355    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255357    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255360    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255363    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255365    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255368    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255370    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255373    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255375    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255378    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255380    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255383    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255386    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255389    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255391    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255394    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255397    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255401    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255403    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 12:42:11.259239 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255406    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255409    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255412    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255415    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255417    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255420    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255423    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255427    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255430    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255432    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255435    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255437    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255440    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255442    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255445    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255447    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255449    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255452    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255454    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255457    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 12:42:11.259713 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255460    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255462    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255465    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255468    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255472    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255474    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255476    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255479    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255482    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255484    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255488    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255491    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255493    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255496    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255498    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255516    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255520    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255523    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255526    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 12:42:11.260230 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255529    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255532    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255535    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255537    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255540    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255543    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255546    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255548    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255551    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255554    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255556    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255559    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255561    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255564    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255567    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255569    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255572    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255574    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255577    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255580    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 12:42:11.260699 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.255582    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255643    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255649    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255656    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255660    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255664    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255667    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255671    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255675    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255678    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255681    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255684    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255688    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255691    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255693    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255696    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255699    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255703    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255706    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255710    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255713    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255716    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255719    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255735    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 12:42:11.261192 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255739    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255742    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255745    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255748    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255751    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255754    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255757    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255760    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255763    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255767    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255769    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255772    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255777    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255780    2579 flags.go:64] FLAG: --enable-server="true"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255783    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255787    2579 flags.go:64] FLAG: --event-burst="100"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255790    2579 flags.go:64] FLAG: --event-qps="50"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255793    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255796    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255799    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255802    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255805    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255808    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255811    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255813    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 12:42:11.261770 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255816    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255819    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255822    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255825    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255827    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255830    2579 flags.go:64] FLAG: --feature-gates=""
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255834    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255836    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255839    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255843    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255845    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255848    2579 flags.go:64] FLAG: --help="false"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255853    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255857    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255859    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255862    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255865    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255869    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255871    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255875    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255878    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255880    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255883    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255886    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 12:42:11.262384 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255889    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255892    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255894    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255897    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255900    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255903    2579 flags.go:64] FLAG: --lock-file=""
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255905    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255908    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255911    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255916    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255919    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255922    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255925    2579 flags.go:64] FLAG: --logging-format="text"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255927    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255930    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255933    2579 flags.go:64] FLAG: --manifest-url=""
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255936    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255940    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255943    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255951    2579 flags.go:64] FLAG: --max-pods="110"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255956    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255959    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255961    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255964    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255967    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 12:42:11.262984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255970    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255973    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255980    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255984    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255987    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255989    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255992    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.255997    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256000    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256003    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256006    2579 flags.go:64] FLAG: --port="10250"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256008    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256011    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08f892bb172a5378c"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256014    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256017    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256020    2579 flags.go:64] FLAG: --register-node="true"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256023    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256026    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256030    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256033    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256035    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256039    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256042    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256045    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256048    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 12:42:11.263584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256050    2579 flags.go:64] FLAG: --runonce="false"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256053    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256056    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256061    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256064    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256066    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256069    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256072    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256075    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256078    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256081    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256083    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256086    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256089    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256092    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256094    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256099    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256102    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256105    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256108    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256111    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256114    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256116    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256119    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256122    2579 flags.go:64] FLAG: --v="2"
Jul 07 12:42:11.264273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256126    2579 flags.go:64] FLAG: --version="false"
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256130    2579 flags.go:64] FLAG: --vmodule=""
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256134    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256137    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256239    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256243    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256246    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256249    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256252    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256256    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256261    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256264    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256266    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256269    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256272    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256275    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256278    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256280    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256283    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 12:42:11.264883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256286    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256288    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256291    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256293    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256296    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256298    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256301    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256304    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256306    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256309    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256311    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256315    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256319    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256322    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256325    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256328    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256331    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256334    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256337    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 12:42:11.265339 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256339    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256342    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256349    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256352    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256356    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256358    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256361    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256363    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256366    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256368    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256371    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256374    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256376    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256379    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256382    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256384    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256387    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256389    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256392    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256394    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 12:42:11.265813 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256397    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256399    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256402    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256404    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256407    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256409    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256411    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256414    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256417    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256421    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256424    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256426    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256429    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256431    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256435    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256438    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256442    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256445    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256447    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256450    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 12:42:11.266311 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256452    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256455    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256457    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256460    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256462    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256465    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256467    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256470    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256472    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256475    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256477    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256480    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256482    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256485    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256487    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256490    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256492    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256495    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256497    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256500    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 12:42:11.266822 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256502    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256505    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256508    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256510    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.256513    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.256520    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.262564    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.262580    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.262599    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.262607    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262658    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262663    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262666    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262669    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262672    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 12:42:11.267319 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262674    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262677    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262680    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262682    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262685    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262688    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262690    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262692    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262695    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262698    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262700    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262702    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262705    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262707    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262710    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262712    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262715    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262717    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262720    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262738    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 12:42:11.267684 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262742    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262745    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262747    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262750    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262754    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262757    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262760    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262762    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262765    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262768    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262771    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262773    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262776    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262778    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262782    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262786    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262789    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262791    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262794    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262797    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 12:42:11.268226 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262799    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262803    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262806    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262809    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262812    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262815    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262818    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262821    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262823    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262826    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262828    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262831    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262834    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262836    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262838    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262841    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262844    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262847    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262850    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262852    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 12:42:11.268718 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262855    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262858    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262860    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262863    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262865    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262868    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262870    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262873    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262875    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262878    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262880    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262883    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262885    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262888    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262890    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262894    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262897    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262900    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262902    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 12:42:11.269211 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262905    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262908    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262911    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262914    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262916    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262919    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262922    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262924    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262927    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262930    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262933    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262936    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262939    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262942    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.262945    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 12:42:11.269692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.262950    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263056    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263062    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263065    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263068    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263071    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263074    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263077    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263079    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263082    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263085    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263087    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263090    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263093    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263096    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263098    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263101    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263104    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263106    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 12:42:11.270194 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263109    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263111    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263114    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263116    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263119    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263121    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263123    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263126    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263128    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263131    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263134    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263136    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263139    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263141    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263143    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263146    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263148    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263151    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263153    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263155    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 12:42:11.270650 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263158    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263160    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263163    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263165    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263168    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263170    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263173    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263175    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263177    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263180    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263182    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263185    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263187    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263190    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263192    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263194    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263197    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263200    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263202    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263205    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 12:42:11.271159 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263207    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263210    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263212    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263215    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263217    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263220    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263222    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263225    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263227    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263229    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263232    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263234    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263237    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263239    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263242    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263246    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263249    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263251    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263254    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263256    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 12:42:11.271652 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263258    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263261    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263264    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263266    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263269    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263273    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263276    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263279    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263282    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263286    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263289    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263292    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263295    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263298    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263301    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263304    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263307    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263309    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263312    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 12:42:11.272367 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263314    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:11.263316    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.263321    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.263408    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.266343    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.267713    2579 server.go:1019] "Starting client certificate rotation"
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.267825    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.267859    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.270362    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 12:42:11.273041 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.272991    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 12:42:11.286042 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.286020    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 07 12:42:11.288173 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.288159    2579 log.go:25] "Validated CRI v1 image API"
Jul 07 12:42:11.289679 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.289665    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 12:42:11.293198 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.293180    2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 82d5b444-b8f9-418d-bbc3-b87e6078af21:/dev/nvme0n1p4 fac4f1bf-5f1f-418d-940a-64e71d4b70e7:/dev/nvme0n1p3]
Jul 07 12:42:11.293300 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.293198    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 12:42:11.298114 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.297999    2579 manager.go:216] Machine: {Timestamp:2026-07-07 12:42:11.296796612 +0000 UTC m=+0.328456249 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101919 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ce9f52de464f97bcf9ca75f7f7910 SystemUUID:ec2ce9f5-2de4-64f9-7bcf-9ca75f7f7910 BootID:a81ec39e-9a2a-4bd7-8b45-50ebb308df8e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:82:d0:5a:2c:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:82:d0:5a:2c:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:42:5b:d9:23:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 12:42:11.298114 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.298105    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 12:42:11.298323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.298186    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 12:42:11.299858 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.299839    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 12:42:11.300960 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.300934    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 12:42:11.301104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.300966    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 12:42:11.301754 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.301705    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 12:42:11.301811 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.301760    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 12:42:11.301811 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.301771    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 12:42:11.302308 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.302297    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 12:42:11.303375 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.303363    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 12:42:11.306125 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306111    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 12:42:11.306246 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306236    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 12:42:11.306302 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306294    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 12:42:11.306353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306309    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 12:42:11.306353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306329    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 12:42:11.306353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306343    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 07 12:42:11.306469 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306355    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 12:42:11.306666 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306653    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 12:42:11.306751 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.306675    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 12:42:11.308635 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.308609    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 12:42:11.310721 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.310706    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 12:42:11.310794 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.310769    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 12:42:11.311761 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311743    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 12:42:11.311761 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311759    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 12:42:11.311761 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311765    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311771    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311781    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311786    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311795    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311800    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311806    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311812    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311820    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 12:42:11.311896 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.311825    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 12:42:11.312612 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.312602    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 12:42:11.312612 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.312611    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 12:42:11.315525 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.315503    2579 server.go:1268] "Started kubelet"
Jul 07 12:42:11.318983 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.318794    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 12:42:11.318983 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.318868    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 12:42:11.318983 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.318894    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 12:42:11.319319 ip-10-0-132-141 systemd[1]: Started Kubernetes Kubelet.
Jul 07 12:42:11.320372 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.320338    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 12:42:11.320850 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.320828    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2nrwv"
Jul 07 12:42:11.322061 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.322037    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 12:42:11.322424 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.322405    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 12:42:11.322510 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.322416    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 12:42:11.325736 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.325707    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 12:42:11.327590 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.327574    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 12:42:11.327679 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.327588    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 12:42:11.327746 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.327706    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:11.328193 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.328177    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 12:42:11.328298 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.328178    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 12:42:11.328298 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.328208    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 12:42:11.328403 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.328322    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 12:42:11.328403 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.328328    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 12:42:11.328993 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.328965    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2nrwv"
Jul 07 12:42:11.329661 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.329645    2579 factory.go:55] Registering systemd factory
Jul 07 12:42:11.329744 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.329689    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 07 12:42:11.329932 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.329919    2579 factory.go:153] Registering CRI-O factory
Jul 07 12:42:11.329989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.329936    2579 factory.go:223] Registration of the crio container factory successfully
Jul 07 12:42:11.329989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.329984    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 12:42:11.330073 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.330009    2579 factory.go:103] Registering Raw factory
Jul 07 12:42:11.330073 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.330023    2579 manager.go:1196] Started watching for new ooms in manager
Jul 07 12:42:11.330998 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.330979    2579 manager.go:349] Starting recovery of all containers
Jul 07 12:42:11.331165 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.331144    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 12:42:11.331259 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.331244    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 12:42:11.331409 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.331390    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 12:42:11.331993 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.331131    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-141.ec2.internal.18c0026fb97f7532  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-141.ec2.internal,UID:ip-10-0-132-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-141.ec2.internal,},FirstTimestamp:2026-07-07 12:42:11.315479858 +0000 UTC m=+0.347139496,LastTimestamp:2026-07-07 12:42:11.315479858 +0000 UTC m=+0.347139496,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-141.ec2.internal,}"
Jul 07 12:42:11.341590 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.341426    2579 manager.go:354] Recovery completed
Jul 07 12:42:11.343014 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.342990    2579 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 07 12:42:11.345765 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.345754    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.347852 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.347839    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.347906 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.347871    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.347906 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.347886    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.348354 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.348339    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 12:42:11.348354 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.348351    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 12:42:11.348468 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.348365    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 12:42:11.348468 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.348386    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.349058 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.349042    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.349107 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.349075    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.349107 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.349090    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.349370 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.349355    2579 policy_none.go:49] "None policy: Start"
Jul 07 12:42:11.349370 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.349372    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 12:42:11.349461 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.349382    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 12:42:11.350200 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.350190    2579 policy_none.go:47] "Start"
Jul 07 12:42:11.383300 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383280    2579 manager.go:337] "Starting Device Plugin manager"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.383317    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383331    2579 server.go:87] "Starting device plugin registration server"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383543    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383552    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383644    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383796    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.383806    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.384102    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 12:42:11.395879 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.384148    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:11.479512 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.479491    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 12:42:11.480627 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.480613    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 12:42:11.480691 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.480645    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 12:42:11.480691 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.480663    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 12:42:11.480791 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.480693    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 12:42:11.483295 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.483278    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:11.484220 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.484208    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.486439 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.486421    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.486524 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.486445    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.486524 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.486456    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.486524 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.486475    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.497045 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.497010    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.497045 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.497029    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-141.ec2.internal\": node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:11.520187 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.520173    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:11.581227 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.581195    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"]
Jul 07 12:42:11.581302 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.581271    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.582644 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.582631    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.582738 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.582657    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.582738 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.582672    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.583925 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.583912    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.584059 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.584101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584075    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.584578 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584562    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.584578 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584576    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.584692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584589    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.584692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584597    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.584692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584603    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.584815 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.584608    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.585695 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.585679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.585788 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.585702    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 12:42:11.586343 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.586328    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 12:42:11.586420 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.586352    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 12:42:11.586420 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.586362    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeHasSufficientPID"
Jul 07 12:42:11.613382 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.613363    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-141.ec2.internal\" not found" node="ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.617834 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.617819    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-141.ec2.internal\" not found" node="ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.620366 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:11.620349    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:11.629505 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.629490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/70e8ef2798f0deb830c2945c57670baa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal\" (UID: \"70e8ef2798f0deb830c2945c57670baa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.629566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.629514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70e8ef2798f0deb830c2945c57670baa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal\" (UID: \"70e8ef2798f0deb830c2945c57670baa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:11.629566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:11.629529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe845dad01dc41ef76e1b98e2c39b5a0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-141.ec2.internal\" (UID: \"fe845dad01dc41ef76e1b98e2c39b5a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.751995 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:12.751980    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:12.761282 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.761268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/70e8ef2798f0deb830c2945c57670baa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal\" (UID: \"70e8ef2798f0deb830c2945c57670baa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.761349 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.761290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70e8ef2798f0deb830c2945c57670baa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal\" (UID: \"70e8ef2798f0deb830c2945c57670baa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.761349 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.761308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe845dad01dc41ef76e1b98e2c39b5a0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-141.ec2.internal\" (UID: \"fe845dad01dc41ef76e1b98e2c39b5a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.761349 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.761347    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe845dad01dc41ef76e1b98e2c39b5a0-config\") pod \"kube-apiserver-proxy-ip-10-0-132-141.ec2.internal\" (UID: \"fe845dad01dc41ef76e1b98e2c39b5a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.761468 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.761381    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70e8ef2798f0deb830c2945c57670baa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal\" (UID: \"70e8ef2798f0deb830c2945c57670baa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.761468 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.761381    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/70e8ef2798f0deb830c2945c57670baa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal\" (UID: \"70e8ef2798f0deb830c2945c57670baa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.852707 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:12.852675    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:12.947238 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.947216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.952254 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:12.952234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:12.953319 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:12.953301    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:13.054177 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.054157    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:13.154703 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.154646    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-141.ec2.internal\" not found"
Jul 07 12:42:13.230890 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.230861    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:13.240159 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.240143    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:13.258875 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.258860    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:13.273779 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.273755    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 12:42:13.274860 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.274848    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal"
Jul 07 12:42:13.283344 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.283328    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 12:42:13.298473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.298452    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 12:42:13.298592 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.298571    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 12:42:13.298592 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.298576    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 12:42:13.298676 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.298577    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 12:42:13.337778 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.337754    2579 apiserver.go:52] "Watching apiserver"
Jul 07 12:42:13.348108 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.348090    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 12:42:13.350082 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.349037    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal","openshift-dns/node-resolver-cs8fp","openshift-image-registry/node-ca-272qj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal","openshift-multus/multus-d8sls","openshift-multus/network-metrics-daemon-f6w76","openshift-network-operator/iptables-alerter-k7hrf","kube-system/konnectivity-agent-6zwxt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9","openshift-cluster-node-tuning-operator/tuned-cwmzh","openshift-multus/multus-additional-cni-plugins-2h4wj","openshift-network-diagnostics/network-check-target-zkr6b","openshift-ovn-kubernetes/ovnkube-node-mk748","kube-system/global-pull-secret-syncer-9ngf2"]
Jul 07 12:42:13.350566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.350550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.351626 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.351606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.352286 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.352269    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.352366 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.352286    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q8jv9\""
Jul 07 12:42:13.352366 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.352275    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.352613 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.352590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.353407 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.353389    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 12:42:13.353660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.353497    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 12:42:13.353660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.353512    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 12:42:13.353660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.353585    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.353660 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.353651    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:13.353894 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.353691    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.354570 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354141    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gfj2n\""
Jul 07 12:42:13.354570 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354195    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.354570 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354287    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 12:42:13.354570 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354372    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.354570 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354378    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.354856 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354595    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hx2h\""
Jul 07 12:42:13.354856 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354606    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 12:42:13.354949 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.354903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.355779 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.355764    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.355876 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.355825    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:13.356705 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.356687    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 12:42:13.356879 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.356846    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g5jtt\""
Jul 07 12:42:13.357133 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.357114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.357226 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.357178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.357538 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.357240    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 12:42:13.357538 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.357119    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.358484 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.358468    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 12:42:13.358583 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.358490    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.358800 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.358785    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 12:42:13.359176 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.359159    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.359262 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.359247    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.359319 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.359253    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nj27s\""
Jul 07 12:42:13.360036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.360010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.360499 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.360482    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 12:42:13.360633 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.360618    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.360771 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.360751    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.360928 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.360912    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlxnj\""
Jul 07 12:42:13.361346 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.361313    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 12:37:12 +0000 UTC" deadline="2027-12-22 07:51:07.761736791 +0000 UTC"
Jul 07 12:42:13.361346 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.361344    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12787h8m54.400394684s"
Jul 07 12:42:13.361637 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.361623    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 12:42:13.361753 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.361739    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 12:42:13.361828 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.361814    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9rv6v\""
Jul 07 12:42:13.364088 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364062    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.364338 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzjpf\" (UniqueName: \"kubernetes.io/projected/aebd87aa-c237-4a76-8d8d-80c8072cd0da-kube-api-access-vzjpf\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.364418 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-run\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364418 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-registration-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.364418 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-systemd\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.364418 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovn-node-metrics-cert\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-cnibin\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-cni-binary-copy\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364469    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-var-lib-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aebd87aa-c237-4a76-8d8d-80c8072cd0da-hosts-file\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebd87aa-c237-4a76-8d8d-80c8072cd0da-tmp-dir\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-daemon-config\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.364611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe45b309-3c33-4709-beda-fae6f97b12b8-iptables-alerter-script\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpczz\" (UniqueName: \"kubernetes.io/projected/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-kube-api-access-rpczz\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-var-lib-kubelet\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd351ee-c98c-4120-b81b-11b12b8558d1-tmp\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364734    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-socket-dir-parent\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-lib-modules\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-tuned\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sdfn\" (UniqueName: \"kubernetes.io/projected/bfd351ee-c98c-4120-b81b-11b12b8558d1-kube-api-access-4sdfn\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364785    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-sys-fs\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-modprobe-d\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-os-release\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364866    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-multus-certs\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-slash\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-env-overrides\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.364973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovnkube-script-lib\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-sys\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-cni-bin\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.364989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wvmk\" (UniqueName: \"kubernetes.io/projected/f1730709-e335-4671-8081-052fcc0072a7-kube-api-access-4wvmk\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-systemd\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365030    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-kubelet\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-kubernetes\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-dbus\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-etc-kubernetes\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-run-netns\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovnkube-config\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365155    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-netns\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m454f\" (UniqueName: \"kubernetes.io/projected/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-kube-api-access-m454f\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365229    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p542q\" (UniqueName: \"kubernetes.io/projected/fe45b309-3c33-4709-beda-fae6f97b12b8-kube-api-access-p542q\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-ovn\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365319    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysconfig\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.365689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ac942c-b28d-40da-b650-0451e463eb97-host\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-socket-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365405    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-host\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-cni-multus\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-hostroot\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f4ac942c-b28d-40da-b650-0451e463eb97-serviceca\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-system-cni-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-cni-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-kubelet-config\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6sbj\" (UniqueName: \"kubernetes.io/projected/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-kube-api-access-c6sbj\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-kubelet\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365646    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-systemd-units\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-etc-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysctl-conf\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365729    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365744    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-device-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.366409 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-log-socket\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysctl-d\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe45b309-3c33-4709-beda-fae6f97b12b8-host-slash\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-cni-bin\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-k8s-cni-cncf-io\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-conf-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.365991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-node-log\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.366025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-cni-netd\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.366048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.366066    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wv4kb\""
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.366071    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrnn\" (UniqueName: \"kubernetes.io/projected/f4ac942c-b28d-40da-b650-0451e463eb97-kube-api-access-2xrnn\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.366105    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.366760    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:13.367055 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.366814    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:13.367618 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.367376    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 12:42:13.367618 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.367473    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 12:42:13.367618 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.367474    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2q22v\""
Jul 07 12:42:13.373137 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.373119    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 12:42:13.393049 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.393031    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d7sjn"
Jul 07 12:42:13.400089 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.400074    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:13.403259 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.403243    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d7sjn"
Jul 07 12:42:13.459692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.459674    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 12:42:13.467186 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-hostroot\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.467283 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f4ac942c-b28d-40da-b650-0451e463eb97-serviceca\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.467283 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-system-cni-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.467283 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-cni-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-kubelet-config\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467316    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-system-cni-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6sbj\" (UniqueName: \"kubernetes.io/projected/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-kube-api-access-c6sbj\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467318    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-hostroot\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467372    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-kubelet-config\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-kubelet\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467414    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-kubelet\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467481    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-cni-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-systemd-units\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-etc-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysctl-conf\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467672    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f4ac942c-b28d-40da-b650-0451e463eb97-serviceca\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-device-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-log-socket\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467750    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-etc-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysctl-d\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.467780 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467775    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysctl-conf\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467792    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-os-release\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467805    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-device-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe45b309-3c33-4709-beda-fae6f97b12b8-host-slash\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467825    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-log-socket\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-cni-bin\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467854    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-k8s-cni-cncf-io\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-conf-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467901    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe45b309-3c33-4709-beda-fae6f97b12b8-host-slash\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467942    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysctl-d\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-node-log\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467944    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-k8s-cni-cncf-io\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467948    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-cni-bin\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.467972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-cni-netd\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468001    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-conf-dir\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.468285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468018    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-cni-netd\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrnn\" (UniqueName: \"kubernetes.io/projected/f4ac942c-b28d-40da-b650-0451e463eb97-kube-api-access-2xrnn\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468059    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468061    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-node-log\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzjpf\" (UniqueName: \"kubernetes.io/projected/aebd87aa-c237-4a76-8d8d-80c8072cd0da-kube-api-access-vzjpf\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468097    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-systemd-units\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-run\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-registration-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-run\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468464    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-registration-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-systemd\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468559    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-systemd\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovn-node-metrics-cert\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-cnibin\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-cni-binary-copy\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-system-cni-dir\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.469075 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468738    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-cnibin\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-var-lib-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aebd87aa-c237-4a76-8d8d-80c8072cd0da-hosts-file\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebd87aa-c237-4a76-8d8d-80c8072cd0da-tmp-dir\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468881    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-var-lib-openvswitch\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-daemon-config\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468900    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.468969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe45b309-3c33-4709-beda-fae6f97b12b8-iptables-alerter-script\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpczz\" (UniqueName: \"kubernetes.io/projected/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-kube-api-access-rpczz\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-var-lib-kubelet\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469056    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aebd87aa-c237-4a76-8d8d-80c8072cd0da-hosts-file\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd351ee-c98c-4120-b81b-11b12b8558d1-tmp\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-socket-dir-parent\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469123    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-lib-modules\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-tuned\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470069 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sdfn\" (UniqueName: \"kubernetes.io/projected/bfd351ee-c98c-4120-b81b-11b12b8558d1-kube-api-access-4sdfn\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-sys-fs\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469263    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-cni-binary-copy\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469327    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-socket-dir-parent\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469336    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469450    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-var-lib-kubelet\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469614    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-lib-modules\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-modprobe-d\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-os-release\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-multus-certs\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-slash\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-env-overrides\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovnkube-script-lib\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.469983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-sys\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-cni-bin\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470006    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe45b309-3c33-4709-beda-fae6f97b12b8-iptables-alerter-script\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.470958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wvmk\" (UniqueName: \"kubernetes.io/projected/f1730709-e335-4671-8081-052fcc0072a7-kube-api-access-4wvmk\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-systemd\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-kubelet\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v6g6\" (UniqueName: \"kubernetes.io/projected/40316917-dd36-48b1-b4ef-acd4c3ab87ab-kube-api-access-4v6g6\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-kubernetes\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470191    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-multus-daemon-config\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-dbus\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-etc-kubernetes\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed74092e-b808-4be1-8b12-c57dae291412-konnectivity-ca\") pod \"konnectivity-agent-6zwxt\" (UID: \"ed74092e-b808-4be1-8b12-c57dae291412\") " pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470247    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebd87aa-c237-4a76-8d8d-80c8072cd0da-tmp-dir\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-run-netns\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovnkube-config\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-netns\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-os-release\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m454f\" (UniqueName: \"kubernetes.io/projected/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-kube-api-access-m454f\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470397    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-sys-fs\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.471772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed74092e-b808-4be1-8b12-c57dae291412-agent-certs\") pod \"konnectivity-agent-6zwxt\" (UID: \"ed74092e-b808-4be1-8b12-c57dae291412\") " pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cnibin\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470455    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-cni-bin\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470573    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-slash\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.470577    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470647    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-modprobe-d\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.470659    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:13.970624893 +0000 UTC m=+1.971491810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470697    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-sys\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470726    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-systemd\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470756    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-kubelet\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470774    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-run-netns\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470789    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovnkube-script-lib\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470816    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-kubernetes\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470824    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-multus-certs\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470824    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-run-netns\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.470916    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-dbus\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.472660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471246    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471317    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-etc-kubernetes\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471585    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-env-overrides\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p542q\" (UniqueName: \"kubernetes.io/projected/fe45b309-3c33-4709-beda-fae6f97b12b8-kube-api-access-p542q\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-ovn\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysconfig\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471772    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovnkube-config\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ac942c-b28d-40da-b650-0451e463eb97-host\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471848    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4ac942c-b28d-40da-b650-0451e463eb97-host\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cni-binary-copy\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-socket-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.471931    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.471972    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-run-ovn\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-host\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-cni-multus\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.473453 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.472135    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:13.972115465 +0000 UTC m=+1.972982379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472169    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-host-var-lib-cni-multus\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472202    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-sysconfig\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472224    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfd351ee-c98c-4120-b81b-11b12b8558d1-host\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472333    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472463    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-socket-dir\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.472607    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bfd351ee-c98c-4120-b81b-11b12b8558d1-etc-tuned\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.474297 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.473886    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd351ee-c98c-4120-b81b-11b12b8558d1-tmp\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.477207 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.476276    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-ovn-node-metrics-cert\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.477207 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.476291    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzjpf\" (UniqueName: \"kubernetes.io/projected/aebd87aa-c237-4a76-8d8d-80c8072cd0da-kube-api-access-vzjpf\") pod \"node-resolver-cs8fp\" (UID: \"aebd87aa-c237-4a76-8d8d-80c8072cd0da\") " pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.477207 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.476594    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrnn\" (UniqueName: \"kubernetes.io/projected/f4ac942c-b28d-40da-b650-0451e463eb97-kube-api-access-2xrnn\") pod \"node-ca-272qj\" (UID: \"f4ac942c-b28d-40da-b650-0451e463eb97\") " pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.477207 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.477083    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6sbj\" (UniqueName: \"kubernetes.io/projected/b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5-kube-api-access-c6sbj\") pod \"aws-ebs-csi-driver-node-r8gm9\" (UID: \"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.478194 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.478173    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sdfn\" (UniqueName: \"kubernetes.io/projected/bfd351ee-c98c-4120-b81b-11b12b8558d1-kube-api-access-4sdfn\") pod \"tuned-cwmzh\" (UID: \"bfd351ee-c98c-4120-b81b-11b12b8558d1\") " pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.478303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.478233    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpczz\" (UniqueName: \"kubernetes.io/projected/f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6-kube-api-access-rpczz\") pod \"ovnkube-node-mk748\" (UID: \"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.479813 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.479787    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wvmk\" (UniqueName: \"kubernetes.io/projected/f1730709-e335-4671-8081-052fcc0072a7-kube-api-access-4wvmk\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.480287 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.480266    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m454f\" (UniqueName: \"kubernetes.io/projected/1a958f0e-ff00-47f5-8351-89aaf5ae69ed-kube-api-access-m454f\") pod \"multus-d8sls\" (UID: \"1a958f0e-ff00-47f5-8351-89aaf5ae69ed\") " pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.482224 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.482203    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe845dad01dc41ef76e1b98e2c39b5a0.slice/crio-5059200580d1927dd108945a7ae892c5c99ca607d834ab99773d231cf2cfe1fe WatchSource:0}: Error finding container 5059200580d1927dd108945a7ae892c5c99ca607d834ab99773d231cf2cfe1fe: Status 404 returned error can't find the container with id 5059200580d1927dd108945a7ae892c5c99ca607d834ab99773d231cf2cfe1fe
Jul 07 12:42:13.482614 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.482598    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e8ef2798f0deb830c2945c57670baa.slice/crio-2b4b78788f5b49e6c2d9691d09e44a744a47862ada89a7662eed78b23dd10b0e WatchSource:0}: Error finding container 2b4b78788f5b49e6c2d9691d09e44a744a47862ada89a7662eed78b23dd10b0e: Status 404 returned error can't find the container with id 2b4b78788f5b49e6c2d9691d09e44a744a47862ada89a7662eed78b23dd10b0e
Jul 07 12:42:13.486776 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.486761    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 12:42:13.487733 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.487711    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p542q\" (UniqueName: \"kubernetes.io/projected/fe45b309-3c33-4709-beda-fae6f97b12b8-kube-api-access-p542q\") pod \"iptables-alerter-k7hrf\" (UID: \"fe45b309-3c33-4709-beda-fae6f97b12b8\") " pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.489776 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.489755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cs8fp"
Jul 07 12:42:13.498591 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.498558    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaebd87aa_c237_4a76_8d8d_80c8072cd0da.slice/crio-afd2c6011e66d7cbfc71abd32f12813121f41fe7419e4e28b36b99102b14a8a4 WatchSource:0}: Error finding container afd2c6011e66d7cbfc71abd32f12813121f41fe7419e4e28b36b99102b14a8a4: Status 404 returned error can't find the container with id afd2c6011e66d7cbfc71abd32f12813121f41fe7419e4e28b36b99102b14a8a4
Jul 07 12:42:13.513948 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.513913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cs8fp" event={"ID":"aebd87aa-c237-4a76-8d8d-80c8072cd0da","Type":"ContainerStarted","Data":"afd2c6011e66d7cbfc71abd32f12813121f41fe7419e4e28b36b99102b14a8a4"}
Jul 07 12:42:13.514924 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.514904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal" event={"ID":"fe845dad01dc41ef76e1b98e2c39b5a0","Type":"ContainerStarted","Data":"5059200580d1927dd108945a7ae892c5c99ca607d834ab99773d231cf2cfe1fe"}
Jul 07 12:42:13.515762 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.515735    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal" event={"ID":"70e8ef2798f0deb830c2945c57670baa","Type":"ContainerStarted","Data":"2b4b78788f5b49e6c2d9691d09e44a744a47862ada89a7662eed78b23dd10b0e"}
Jul 07 12:42:13.573092 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-system-cni-dir\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v6g6\" (UniqueName: \"kubernetes.io/projected/40316917-dd36-48b1-b4ef-acd4c3ab87ab-kube-api-access-4v6g6\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed74092e-b808-4be1-8b12-c57dae291412-konnectivity-ca\") pod \"konnectivity-agent-6zwxt\" (UID: \"ed74092e-b808-4be1-8b12-c57dae291412\") " pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.573351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573186    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-system-cni-dir\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:13.573351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed74092e-b808-4be1-8b12-c57dae291412-agent-certs\") pod \"konnectivity-agent-6zwxt\" (UID: \"ed74092e-b808-4be1-8b12-c57dae291412\") " pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.573351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573273    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cnibin\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573350    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cnibin\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cni-binary-copy\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-os-release\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573504    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40316917-dd36-48b1-b4ef-acd4c3ab87ab-os-release\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573930 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573806    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed74092e-b808-4be1-8b12-c57dae291412-konnectivity-ca\") pod \"konnectivity-agent-6zwxt\" (UID: \"ed74092e-b808-4be1-8b12-c57dae291412\") " pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.573930 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573883    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.573999 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.573966    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.574032 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.574007    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40316917-dd36-48b1-b4ef-acd4c3ab87ab-cni-binary-copy\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.575303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.575287    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed74092e-b808-4be1-8b12-c57dae291412-agent-certs\") pod \"konnectivity-agent-6zwxt\" (UID: \"ed74092e-b808-4be1-8b12-c57dae291412\") " pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.580482 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.580465    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:13.580482 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.580482    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:13.580612 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.580491    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:13.580612 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.580579    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:14.080560677 +0000 UTC m=+2.081427588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:13.582256 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.582240    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v6g6\" (UniqueName: \"kubernetes.io/projected/40316917-dd36-48b1-b4ef-acd4c3ab87ab-kube-api-access-4v6g6\") pod \"multus-additional-cni-plugins-2h4wj\" (UID: \"40316917-dd36-48b1-b4ef-acd4c3ab87ab\") " pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.680678 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.680631    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cwmzh"
Jul 07 12:42:13.688255 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.688230    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd351ee_c98c_4120_b81b_11b12b8558d1.slice/crio-a98a436a9252d124b531d40e90a6f2ef1207fb902ead9e1a2c0fc8c8ca9be7a5 WatchSource:0}: Error finding container a98a436a9252d124b531d40e90a6f2ef1207fb902ead9e1a2c0fc8c8ca9be7a5: Status 404 returned error can't find the container with id a98a436a9252d124b531d40e90a6f2ef1207fb902ead9e1a2c0fc8c8ca9be7a5
Jul 07 12:42:13.695236 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.695219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:13.701409 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.701383    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43a7d5b_121e_44ff_8fb4_4f0cd2a66bb6.slice/crio-f5a27ac967ebb9172b2966ec49563e4b0f21589e3fac0d7078a1045e9dbf307e WatchSource:0}: Error finding container f5a27ac967ebb9172b2966ec49563e4b0f21589e3fac0d7078a1045e9dbf307e: Status 404 returned error can't find the container with id f5a27ac967ebb9172b2966ec49563e4b0f21589e3fac0d7078a1045e9dbf307e
Jul 07 12:42:13.710450 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.710434    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-272qj"
Jul 07 12:42:13.716609 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.716589    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ac942c_b28d_40da_b650_0451e463eb97.slice/crio-4693bca7204495e2cdd7848113dffbee4ff8656922ab496f099ecce85ba5c1f0 WatchSource:0}: Error finding container 4693bca7204495e2cdd7848113dffbee4ff8656922ab496f099ecce85ba5c1f0: Status 404 returned error can't find the container with id 4693bca7204495e2cdd7848113dffbee4ff8656922ab496f099ecce85ba5c1f0
Jul 07 12:42:13.722628 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.722611    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d8sls"
Jul 07 12:42:13.728498 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.728476    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a958f0e_ff00_47f5_8351_89aaf5ae69ed.slice/crio-a76ff5756c0a446e7444ada6643f8b5f7ee4218b36de9d604a79c0f3e9290121 WatchSource:0}: Error finding container a76ff5756c0a446e7444ada6643f8b5f7ee4218b36de9d604a79c0f3e9290121: Status 404 returned error can't find the container with id a76ff5756c0a446e7444ada6643f8b5f7ee4218b36de9d604a79c0f3e9290121
Jul 07 12:42:13.736460 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.736446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9"
Jul 07 12:42:13.742395 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.742376    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d887b4_0f1b_46ed_a4e3_c0cd134dc8e5.slice/crio-322cc2c088a3dd2f9bbe6e08aa874225aeb3cb6a9ac1c871abe7b0c445977fc0 WatchSource:0}: Error finding container 322cc2c088a3dd2f9bbe6e08aa874225aeb3cb6a9ac1c871abe7b0c445977fc0: Status 404 returned error can't find the container with id 322cc2c088a3dd2f9bbe6e08aa874225aeb3cb6a9ac1c871abe7b0c445977fc0
Jul 07 12:42:13.758493 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.758470    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k7hrf"
Jul 07 12:42:13.764759 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.764742    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe45b309_3c33_4709_beda_fae6f97b12b8.slice/crio-b52f21fdb800eee1cba52c53ca3801870db1df6ecd51ba28a929a34b7b9c8ed1 WatchSource:0}: Error finding container b52f21fdb800eee1cba52c53ca3801870db1df6ecd51ba28a929a34b7b9c8ed1: Status 404 returned error can't find the container with id b52f21fdb800eee1cba52c53ca3801870db1df6ecd51ba28a929a34b7b9c8ed1
Jul 07 12:42:13.796042 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.796026    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:13.800601 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.800583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2h4wj"
Jul 07 12:42:13.802154 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.802138    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded74092e_b808_4be1_8b12_c57dae291412.slice/crio-df856b5188caf761d9070bacdcf1c80b968b83fe3a5211c70db91d2ad71d4143 WatchSource:0}: Error finding container df856b5188caf761d9070bacdcf1c80b968b83fe3a5211c70db91d2ad71d4143: Status 404 returned error can't find the container with id df856b5188caf761d9070bacdcf1c80b968b83fe3a5211c70db91d2ad71d4143
Jul 07 12:42:13.807115 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:13.807100    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40316917_dd36_48b1_b4ef_acd4c3ab87ab.slice/crio-fe1adb9f85b492a8242414934ce91ba3ffc64446dc76f78ced874001311a9ef2 WatchSource:0}: Error finding container fe1adb9f85b492a8242414934ce91ba3ffc64446dc76f78ced874001311a9ef2: Status 404 returned error can't find the container with id fe1adb9f85b492a8242414934ce91ba3ffc64446dc76f78ced874001311a9ef2
Jul 07 12:42:13.975949 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.975867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:13.976101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:13.975940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:13.976101 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.976068    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:13.976101 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.976080    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:13.976253 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.976138    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:14.976118512 +0000 UTC m=+2.976985410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:13.976253 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:13.976157    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:14.97614795 +0000 UTC m=+2.977014851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:14.178153 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.178110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:14.178346 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.178293    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:14.178346 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.178316    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:14.178346 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.178329    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:14.178570 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.178380    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:15.178361916 +0000 UTC m=+3.179228831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:14.405931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.405807    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:37:13 +0000 UTC" deadline="2028-03-05 23:33:20.165885227 +0000 UTC"
Jul 07 12:42:14.405931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.405841    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14578h51m5.760047933s"
Jul 07 12:42:14.474440 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.474237    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:14.511999 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.511974    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:14.512133 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.512097    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:14.543153 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.543107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerStarted","Data":"fe1adb9f85b492a8242414934ce91ba3ffc64446dc76f78ced874001311a9ef2"}
Jul 07 12:42:14.553990 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.553905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"f5a27ac967ebb9172b2966ec49563e4b0f21589e3fac0d7078a1045e9dbf307e"}
Jul 07 12:42:14.558760 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.558732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6zwxt" event={"ID":"ed74092e-b808-4be1-8b12-c57dae291412","Type":"ContainerStarted","Data":"df856b5188caf761d9070bacdcf1c80b968b83fe3a5211c70db91d2ad71d4143"}
Jul 07 12:42:14.581363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.581298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k7hrf" event={"ID":"fe45b309-3c33-4709-beda-fae6f97b12b8","Type":"ContainerStarted","Data":"b52f21fdb800eee1cba52c53ca3801870db1df6ecd51ba28a929a34b7b9c8ed1"}
Jul 07 12:42:14.605645 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.605418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9" event={"ID":"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5","Type":"ContainerStarted","Data":"322cc2c088a3dd2f9bbe6e08aa874225aeb3cb6a9ac1c871abe7b0c445977fc0"}
Jul 07 12:42:14.630150 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.630121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d8sls" event={"ID":"1a958f0e-ff00-47f5-8351-89aaf5ae69ed","Type":"ContainerStarted","Data":"a76ff5756c0a446e7444ada6643f8b5f7ee4218b36de9d604a79c0f3e9290121"}
Jul 07 12:42:14.637365 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.637338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-272qj" event={"ID":"f4ac942c-b28d-40da-b650-0451e463eb97","Type":"ContainerStarted","Data":"4693bca7204495e2cdd7848113dffbee4ff8656922ab496f099ecce85ba5c1f0"}
Jul 07 12:42:14.654572 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.654467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cwmzh" event={"ID":"bfd351ee-c98c-4120-b81b-11b12b8558d1","Type":"ContainerStarted","Data":"a98a436a9252d124b531d40e90a6f2ef1207fb902ead9e1a2c0fc8c8ca9be7a5"}
Jul 07 12:42:14.987664 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.987628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:14.987898 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:14.987707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:14.987898 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.987816    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:14.987898 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.987869    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:16.987852372 +0000 UTC m=+4.988719275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:14.988405 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.988246    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:14.988405 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:14.988297    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:16.988280815 +0000 UTC m=+4.989147713 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:15.189820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:15.189008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:15.189820 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:15.189165    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:15.189820 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:15.189183    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:15.189820 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:15.189195    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:15.189820 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:15.189246    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:17.189229734 +0000 UTC m=+5.190096631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:15.410755 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:15.410692    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:37:13 +0000 UTC" deadline="2028-04-11 10:25:28.324336254 +0000 UTC"
Jul 07 12:42:15.410755 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:15.410752    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15453h43m12.913588688s"
Jul 07 12:42:15.512530 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:15.511837    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:15.512530 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:15.511955    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:15.512530 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:15.512332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:15.512530 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:15.512437    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:16.049328 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:16.049294    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:16.183101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:16.182884    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 12:42:16.515247 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:16.514780    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:16.515247 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:16.514905    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:17.004985 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:17.004950    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:17.005149 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:17.005029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:17.005149 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.005121    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:17.005272 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.005158    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:17.005272 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.005191    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:21.005169337 +0000 UTC m=+9.006036241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:17.005272 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.005212    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:21.005201334 +0000 UTC m=+9.006068231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:17.208232 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:17.207537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:17.208232 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.207745    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:17.208232 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.207769    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:17.208232 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.207783    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:17.208232 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.207838    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:21.207818543 +0000 UTC m=+9.208685455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:17.512676 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:17.512604    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:17.512830 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:17.512611    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:17.512830 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.512743    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:17.512830 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:17.512784    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:18.511742 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:18.511694    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:18.512212 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:18.511825    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:19.512120 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:19.512079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:19.512585 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:19.512226    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:19.512585 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:19.512079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:19.512585 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:19.512505    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:20.512977 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:20.512681    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:20.513435 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:20.513006    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:21.038288 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:21.037977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:21.038288 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:21.038073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:21.038288 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.038198    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:21.038288 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.038238    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:21.038288 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.038263    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:29.038243852 +0000 UTC m=+17.039110774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:21.038288 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.038281    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:29.038271479 +0000 UTC m=+17.039138376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:21.239932 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:21.239895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:21.240127 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.240094    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:21.240127 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.240116    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:21.240238 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.240130    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:21.240238 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.240191    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:29.240171597 +0000 UTC m=+17.241038518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:21.512773 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:21.512647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:21.512956 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.512778    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:21.512956 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:21.512842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:21.513344 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:21.512966    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:22.513981 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:22.513512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:22.513981 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:22.513640    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:23.512588 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:23.512559    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:23.512588 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:23.512579    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:23.512817 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:23.512684    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:23.512871 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:23.512827    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:24.511779 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:24.511746    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:24.512133 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:24.511876    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:25.512419 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:25.512366    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:25.512419 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:25.512407    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:25.512746 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:25.512484    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:25.512746 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:25.512594    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:26.512115 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:26.512080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:26.512286 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:26.512206    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:27.512639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:27.512609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:27.512639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:27.512622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:27.513139 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:27.512739    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:27.513139 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:27.512858    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:28.512474 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:28.512439    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:28.512664 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:28.512598    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:29.100166 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:29.100134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:29.100338 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:29.100179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:29.100338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.100262    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:29.100338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.100263    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:29.100338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.100317    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.100301572 +0000 UTC m=+33.101168469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:29.100338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.100332    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.100324577 +0000 UTC m=+33.101191474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:29.301646 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:29.301612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:29.301851 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.301787    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:29.301851 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.301805    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:29.301851 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.301816    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:29.301979 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.301871    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.301853839 +0000 UTC m=+33.302720757 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:29.512578 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:29.512493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:29.512784 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:29.512493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:29.512784 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.512624    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:29.512784 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:29.512738    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:30.512193 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:30.512154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:30.512405 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:30.512278    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:31.511915 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:31.511873    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:31.511915 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:31.511916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:31.512368 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:31.512023    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:31.512368 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:31.512174    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:32.512926 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:32.512889    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:32.513357 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:32.513003    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:33.512062 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:33.512032    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:33.512198 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:33.512069    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:33.512198 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:33.512170    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:33.512312 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:33.512285    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:34.512116 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.511963    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:34.512619 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:34.512216    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:34.697639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.697605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d8sls" event={"ID":"1a958f0e-ff00-47f5-8351-89aaf5ae69ed","Type":"ContainerStarted","Data":"9d266e64b70c9ab59886b14ddce1ebd5ce356c8990ccb55bc8e9f9aabc1ba607"}
Jul 07 12:42:34.698996 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.698970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cwmzh" event={"ID":"bfd351ee-c98c-4120-b81b-11b12b8558d1","Type":"ContainerStarted","Data":"4e73624229cbef72641b9bfae585e00d683abf2b408299bf48842a8cbebcaf14"}
Jul 07 12:42:34.701380 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.701357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal" event={"ID":"fe845dad01dc41ef76e1b98e2c39b5a0","Type":"ContainerStarted","Data":"b574e52eb4617d23eef6f471b7b702a4f0518e03bfcb6bdc9c4a174e3b4841ff"}
Jul 07 12:42:34.704577 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.704557    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"5dc7a43f6874bcb75757da62a94a28dfcd400456f64097dda355939f27d6341a"}
Jul 07 12:42:34.704658 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.704617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"913e549c05585eb81cad0589b7dfd0e4223c257f3617eb7488385631d990354a"}
Jul 07 12:42:34.704658 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.704647    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"82106bde58b94b47baadf4f0367369d05362f140e6aef8b8eeaec7e28604e4a0"}
Jul 07 12:42:34.704728 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.704673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"d90542c84827f6525ee7a56175abbdd4debb644fcd6e4931e554f44e8d1f789c"}
Jul 07 12:42:34.704728 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.704701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"2f8955bf55a04f777cb0ba0820423970901540af3571c95c8d50993a39e81eda"}
Jul 07 12:42:34.704786 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.704730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"c79c7e9245f39b5d81f693965c8dfac1eeb41a5130d61cd2d2c48f5dc20212ef"}
Jul 07 12:42:34.714368 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.714326    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d8sls" podStartSLOduration=2.752636844 podStartE2EDuration="22.714313272s" totalImagesPullingTime="19.961676428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:34.713690565 +0000 UTC m=+22.714557485" watchObservedRunningTime="2026-07-07 12:42:34.714313272 +0000 UTC m=+22.715180191"
Jul 07 12:42:34.740228 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:34.740174    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-141.ec2.internal" podStartSLOduration=21.740161977 podStartE2EDuration="21.740161977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:34.725261057 +0000 UTC m=+22.726127976" watchObservedRunningTime="2026-07-07 12:42:34.740161977 +0000 UTC m=+22.741028895"
Jul 07 12:42:35.512357 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.512190    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:35.512830 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.512240    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:35.512830 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:35.512433    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:35.512830 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:35.512507    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:35.707377 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.707345    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cs8fp" event={"ID":"aebd87aa-c237-4a76-8d8d-80c8072cd0da","Type":"ContainerStarted","Data":"9a373f0bc7353bb60fcb71fcc071770190aa7dc6da4230f9d6867976fab6db1a"}
Jul 07 12:42:35.708690 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.708666    2579 generic.go:356] "Generic (PLEG): container finished" podID="70e8ef2798f0deb830c2945c57670baa" containerID="4c61016c6c737ed4253d6cbb01e69cf7f6215eea603a03661116e1d1a9decef2" exitCode=0
Jul 07 12:42:35.708807 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.708742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal" event={"ID":"70e8ef2798f0deb830c2945c57670baa","Type":"ContainerDied","Data":"4c61016c6c737ed4253d6cbb01e69cf7f6215eea603a03661116e1d1a9decef2"}
Jul 07 12:42:35.710152 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.710131    2579 generic.go:356] "Generic (PLEG): container finished" podID="40316917-dd36-48b1-b4ef-acd4c3ab87ab" containerID="95f03a12b9a0d0a05f80c217f86ac704823cec56d8ab39761c39703d5d6c232e" exitCode=0
Jul 07 12:42:35.710265 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.710155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerDied","Data":"95f03a12b9a0d0a05f80c217f86ac704823cec56d8ab39761c39703d5d6c232e"}
Jul 07 12:42:35.711878 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.711852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6zwxt" event={"ID":"ed74092e-b808-4be1-8b12-c57dae291412","Type":"ContainerStarted","Data":"8515780fa2800f86ea443278478fd156b99315ab16a286aa83f90b7e22ae1c78"}
Jul 07 12:42:35.713241 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.713221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k7hrf" event={"ID":"fe45b309-3c33-4709-beda-fae6f97b12b8","Type":"ContainerStarted","Data":"cb5eed3a2633c4c81ea45e9073a778b7cc52e39d06aa33fe2db73cb3cb397b95"}
Jul 07 12:42:35.714728 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.714665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9" event={"ID":"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5","Type":"ContainerStarted","Data":"7e65a86aba7531f057b3482e8c0f89feb1bb222771512d93df73f8301bd5237f"}
Jul 07 12:42:35.716146 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.716122    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-272qj" event={"ID":"f4ac942c-b28d-40da-b650-0451e463eb97","Type":"ContainerStarted","Data":"47267af3e66f52726a380191c255bacf8daffd33fa3eb98f6995202c26e5d11f"}
Jul 07 12:42:35.721243 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.721203    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cwmzh" podStartSLOduration=3.732766677 podStartE2EDuration="23.721190577s" totalImagesPullingTime="19.9884239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:34.748337964 +0000 UTC m=+22.749204885" watchObservedRunningTime="2026-07-07 12:42:35.721190577 +0000 UTC m=+23.722057490"
Jul 07 12:42:35.721724 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.721695    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cs8fp" podStartSLOduration=3.567216788 podStartE2EDuration="23.721687762s" totalImagesPullingTime="20.154470974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:35.721430132 +0000 UTC m=+23.722297053" watchObservedRunningTime="2026-07-07 12:42:35.721687762 +0000 UTC m=+23.722554681"
Jul 07 12:42:35.748905 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.745821    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-272qj" podStartSLOduration=11.987087418 podStartE2EDuration="23.745807393s" totalImagesPullingTime="11.758719975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:35.745432302 +0000 UTC m=+23.746299212" watchObservedRunningTime="2026-07-07 12:42:35.745807393 +0000 UTC m=+23.746674303"
Jul 07 12:42:35.761942 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.761893    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k7hrf" podStartSLOduration=3.8522022099999997 podStartE2EDuration="23.761881037s" totalImagesPullingTime="19.909678827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:35.761715872 +0000 UTC m=+23.762582791" watchObservedRunningTime="2026-07-07 12:42:35.761881037 +0000 UTC m=+23.762747954"
Jul 07 12:42:35.775240 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:35.775152    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6zwxt" podStartSLOduration=3.924730266 podStartE2EDuration="23.775139683s" totalImagesPullingTime="19.850409417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:35.774900678 +0000 UTC m=+23.775767598" watchObservedRunningTime="2026-07-07 12:42:35.775139683 +0000 UTC m=+23.776006603"
Jul 07 12:42:36.487199 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.487172    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 12:42:36.512243 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.512221    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:36.512370 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:36.512347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:36.720831 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.720793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal" event={"ID":"70e8ef2798f0deb830c2945c57670baa","Type":"ContainerStarted","Data":"aedde033c93a1304d765c425633ace7922fc3d7f439ddd17ab96465a32b72b51"}
Jul 07 12:42:36.722948 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.722920    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9" event={"ID":"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5","Type":"ContainerStarted","Data":"4980f016975e6b78214fb3c45a482747210de7358153082d5761f44ce4729c6e"}
Jul 07 12:42:36.734852 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.734807    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-141.ec2.internal" podStartSLOduration=23.734790624 podStartE2EDuration="23.734790624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:36.734762154 +0000 UTC m=+24.735629075" watchObservedRunningTime="2026-07-07 12:42:36.734790624 +0000 UTC m=+24.735657545"
Jul 07 12:42:36.866163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.866138    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:36.866755 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:36.866733    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:37.426487 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:37.426392    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T12:42:36.487195178Z","UUID":"7eb598aa-8d55-413c-9d97-12a54ec61a60","Handler":null,"Name":"","Endpoint":""}
Jul 07 12:42:37.428145 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:37.427984    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 12:42:37.428286 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:37.428169    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 12:42:37.512090 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:37.512067    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:37.512253 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:37.512190    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:37.512253 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:37.512240    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:37.512379 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:37.512344    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:37.731771 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:37.731700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"0bb1d76eaf88c297a5d175e05f88e29bc499e7765bbbd2f919f7abfde62ed43d"}
Jul 07 12:42:38.512651 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:38.512597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:38.512847 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:38.512730    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:38.736346 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:38.736300    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9" event={"ID":"b3d887b4-0f1b-46ed-a4e3-c0cd134dc8e5","Type":"ContainerStarted","Data":"14e48c870f3d95dfd02c6062a5b63a7ca6dff9e5178ab9d09d145eaef6da11c1"}
Jul 07 12:42:38.736763 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:38.736322    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 12:42:38.753925 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:38.753854    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8gm9" podStartSLOduration=4.095186548 podStartE2EDuration="26.753837245s" totalImagesPullingTime="22.658650697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:38.753416319 +0000 UTC m=+26.754283248" watchObservedRunningTime="2026-07-07 12:42:38.753837245 +0000 UTC m=+26.754704166"
Jul 07 12:42:39.512335 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:39.512308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:39.512335 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:39.512330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:39.512565 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:39.512420    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:39.512626 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:39.512561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:40.512135 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:40.512020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:40.512135 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:40.512127    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:40.744176 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:40.743990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" event={"ID":"f43a7d5b-121e-44ff-8fb4-4f0cd2a66bb6","Type":"ContainerStarted","Data":"ab70e78fcbe9d84de3dcf4704552fe015a9f6217fd66d5fd05072f1d6addd40c"}
Jul 07 12:42:40.744288 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:40.744237    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:40.744288 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:40.744288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:40.761585 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:40.761566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:40.770046 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:40.769998    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mk748" podStartSLOduration=8.416989881 podStartE2EDuration="28.769980212s" totalImagesPullingTime="20.352990331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:40.769068307 +0000 UTC m=+28.769935228" watchObservedRunningTime="2026-07-07 12:42:40.769980212 +0000 UTC m=+28.770847131"
Jul 07 12:42:41.114345 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.114320    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:41.114506 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.114456    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 12:42:41.114989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.114975    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6zwxt"
Jul 07 12:42:41.512232 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.512167    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:41.512643 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.512167    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:41.512643 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:41.512258    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:41.512643 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:41.512352    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:41.749157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.749122    2579 generic.go:356] "Generic (PLEG): container finished" podID="40316917-dd36-48b1-b4ef-acd4c3ab87ab" containerID="71ec9107da6288e4b1be3dce3037b7d1ef27598bc50063464c26d4995dc06a69" exitCode=0
Jul 07 12:42:41.749950 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.749255    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerDied","Data":"71ec9107da6288e4b1be3dce3037b7d1ef27598bc50063464c26d4995dc06a69"}
Jul 07 12:42:41.752371 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.752287    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:41.776363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:41.776294    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:42:42.513187 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.513156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:42.513560 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:42.513234    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:42.539787 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.539760    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkr6b"]
Jul 07 12:42:42.543699 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.543676    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9ngf2"]
Jul 07 12:42:42.543799 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.543766    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:42.543874 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:42.543843    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:42.558349 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.558325    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f6w76"]
Jul 07 12:42:42.558445 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.558432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:42.558590 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:42.558561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:42.752360 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:42.752333    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:42.752509 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:42.752477    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:43.643948 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:43.643766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:43.737048 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:43.737021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:43.756450 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:43.756427    2579 generic.go:356] "Generic (PLEG): container finished" podID="40316917-dd36-48b1-b4ef-acd4c3ab87ab" containerID="7c3c8dcd2830ae66b1ace1d4bb80301206e10e52c3345e0f102e01da0c5577b2" exitCode=0
Jul 07 12:42:43.756576 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:43.756528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerDied","Data":"7c3c8dcd2830ae66b1ace1d4bb80301206e10e52c3345e0f102e01da0c5577b2"}
Jul 07 12:42:44.512407 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:44.512375    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:44.512407 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:44.512375    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:44.512661 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:44.512543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:44.512661 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:44.512545    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:44.512771 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:44.512670    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:44.512771 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:44.512755    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:44.761214 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:44.761172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerStarted","Data":"640b7cfb2c23c5ae7aa0d9d745071631394138b4a516d921385551435e457434"}
Jul 07 12:42:45.128479 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:45.128450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:45.128690 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:45.128492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:45.128690 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.128583    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:45.128690 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.128602    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:45.128690 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.128626    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret podName:2a0476f5-c151-4bec-9b97-adc1b69ea8b5 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:17.128613694 +0000 UTC m=+65.129480591 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret") pod "global-pull-secret-syncer-9ngf2" (UID: "2a0476f5-c151-4bec-9b97-adc1b69ea8b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 12:42:45.128690 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.128653    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs podName:f1730709-e335-4671-8081-052fcc0072a7 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:17.128635213 +0000 UTC m=+65.129502123 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs") pod "network-metrics-daemon-f6w76" (UID: "f1730709-e335-4671-8081-052fcc0072a7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 12:42:45.330035 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:45.330005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:45.330190 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.330128    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 12:42:45.330190 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.330144    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 12:42:45.330190 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.330155    2579 projected.go:196] Error preparing data for projected volume kube-api-access-qjwpr for pod openshift-network-diagnostics/network-check-target-zkr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:45.330290 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:45.330196    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr podName:9f0183ef-0993-4e30-87d9-64c72faee949 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:17.330184641 +0000 UTC m=+65.331051550 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qjwpr" (UniqueName: "kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr") pod "network-check-target-zkr6b" (UID: "9f0183ef-0993-4e30-87d9-64c72faee949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 12:42:45.765958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:45.765923    2579 generic.go:356] "Generic (PLEG): container finished" podID="40316917-dd36-48b1-b4ef-acd4c3ab87ab" containerID="640b7cfb2c23c5ae7aa0d9d745071631394138b4a516d921385551435e457434" exitCode=0
Jul 07 12:42:45.766329 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:45.765976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerDied","Data":"640b7cfb2c23c5ae7aa0d9d745071631394138b4a516d921385551435e457434"}
Jul 07 12:42:46.511974 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:46.511945    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:46.512144 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:46.511976    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:46.512144 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:46.511946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:46.512144 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:46.512059    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9ngf2" podUID="2a0476f5-c151-4bec-9b97-adc1b69ea8b5"
Jul 07 12:42:46.512144 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:46.512132    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f6w76" podUID="f1730709-e335-4671-8081-052fcc0072a7"
Jul 07 12:42:46.512344 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:46.512205    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkr6b" podUID="9f0183ef-0993-4e30-87d9-64c72faee949"
Jul 07 12:42:47.760278 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.760190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:47.762559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.762542    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-141.ec2.internal" event="NodeReady"
Jul 07 12:42:47.762673 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.762662    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 12:42:47.805090 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.804869    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gprxp"]
Jul 07 12:42:47.808806 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.808771    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c5h8s"]
Jul 07 12:42:47.808963 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.808910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:47.810944 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.810882    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 12:42:47.810944 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.810939    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zphhp\""
Jul 07 12:42:47.811153 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.811069    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 12:42:47.811270 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.811254    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 12:42:47.812330 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.812308    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69b67766c6-24kgp"]
Jul 07 12:42:47.812477 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.812455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:47.814097 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.814077    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 12:42:47.814202 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.814138    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 12:42:47.814304 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.814278    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ncl7q\""
Jul 07 12:42:47.815934 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.815911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.817692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.817675    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 12:42:47.817800 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.817698    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8ss65\""
Jul 07 12:42:47.817800 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.817788    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 12:42:47.817918 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.817788    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 12:42:47.823087 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.823061    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gprxp"]
Jul 07 12:42:47.823334 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.823314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c5h8s"]
Jul 07 12:42:47.824078 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.824058    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69b67766c6-24kgp"]
Jul 07 12:42:47.824682 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.824662    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 12:42:47.827819 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.827793    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:47.931182 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.931129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:47.951886 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.951858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-certificates\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.951908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzf6p\" (UniqueName: \"kubernetes.io/projected/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-kube-api-access-fzf6p\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:47.952002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.951934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-image-registry-private-configuration\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.951960    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zqwg\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-kube-api-access-7zqwg\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.951983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.951999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-trusted-ca\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952219 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-config-volume\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:47.952219 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-installation-pull-secrets\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952219 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6863a821-1fd3-42d2-9ecf-17ecfc19d394-ca-trust-extracted\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952219 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-bound-sa-token\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:47.952219 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f-cert\") pod \"ingress-canary-gprxp\" (UID: \"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f\") " pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:47.952408 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwcx6\" (UniqueName: \"kubernetes.io/projected/d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f-kube-api-access-gwcx6\") pod \"ingress-canary-gprxp\" (UID: \"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f\") " pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:47.952408 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:47.952408 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:47.952329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-tmp-dir\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.053285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f-cert\") pod \"ingress-canary-gprxp\" (UID: \"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f\") " pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:48.053285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwcx6\" (UniqueName: \"kubernetes.io/projected/d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f-kube-api-access-gwcx6\") pod \"ingress-canary-gprxp\" (UID: \"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f\") " pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:48.053285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.053565 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.053382    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 12:42:48.053565 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-tmp-dir\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.053565 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.053454    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls podName:055e0d3d-3f8f-4d6a-acb3-7372795eda3f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:48.553433731 +0000 UTC m=+36.554300627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls") pod "dns-default-c5h8s" (UID: "055e0d3d-3f8f-4d6a-acb3-7372795eda3f") : secret "dns-default-metrics-tls" not found
Jul 07 12:42:48.053565 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-certificates\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053565 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzf6p\" (UniqueName: \"kubernetes.io/projected/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-kube-api-access-fzf6p\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-image-registry-private-configuration\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zqwg\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-kube-api-access-7zqwg\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-trusted-ca\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-config-volume\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-installation-pull-secrets\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6863a821-1fd3-42d2-9ecf-17ecfc19d394-ca-trust-extracted\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053771    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-tmp-dir\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.053816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.053782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-bound-sa-token\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.054473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.054363    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6863a821-1fd3-42d2-9ecf-17ecfc19d394-ca-trust-extracted\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.054473 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.054377    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 12:42:48.054473 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.054395    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b67766c6-24kgp: secret "image-registry-tls" not found
Jul 07 12:42:48.054473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.054408    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-certificates\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.054473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.054379    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-config-volume\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.054473 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.054453    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls podName:6863a821-1fd3-42d2-9ecf-17ecfc19d394 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:48.554438955 +0000 UTC m=+36.555305875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls") pod "image-registry-69b67766c6-24kgp" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394") : secret "image-registry-tls" not found
Jul 07 12:42:48.054772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.054750    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-trusted-ca\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.058232 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.058043    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-installation-pull-secrets\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.058232 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.058160    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f-cert\") pod \"ingress-canary-gprxp\" (UID: \"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f\") " pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:48.058568 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.058547    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-image-registry-private-configuration\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.065876 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.065237    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zqwg\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-kube-api-access-7zqwg\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.065876 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.065635    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-bound-sa-token\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.066062 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.066044    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzf6p\" (UniqueName: \"kubernetes.io/projected/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-kube-api-access-fzf6p\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.066912 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.066886    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwcx6\" (UniqueName: \"kubernetes.io/projected/d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f-kube-api-access-gwcx6\") pod \"ingress-canary-gprxp\" (UID: \"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f\") " pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:48.122591 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.122555    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gprxp"
Jul 07 12:42:48.271620 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.271592    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gprxp"]
Jul 07 12:42:48.275898 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:48.275871    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd94a46e3_a622_44a6_9fb2_d4bdaf2c0e2f.slice/crio-de6668048c62f3defbc77431b8dc3b293c3165984a191de6c9da68183e8d30dd WatchSource:0}: Error finding container de6668048c62f3defbc77431b8dc3b293c3165984a191de6c9da68183e8d30dd: Status 404 returned error can't find the container with id de6668048c62f3defbc77431b8dc3b293c3165984a191de6c9da68183e8d30dd
Jul 07 12:42:48.512989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.512533    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:42:48.512989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.512542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:42:48.512989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.512691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:42:48.515323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.515119    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 12:42:48.515323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.515132    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 12:42:48.515323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.515140    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 12:42:48.515323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.515215    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-849mb\""
Jul 07 12:42:48.515323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.515162    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gxjtk\""
Jul 07 12:42:48.515323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.515287    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 12:42:48.558002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.557971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:48.558197 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.558042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:48.558197 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.558148    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 12:42:48.558197 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.558170    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b67766c6-24kgp: secret "image-registry-tls" not found
Jul 07 12:42:48.558338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.558206    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 12:42:48.558338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.558235    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls podName:6863a821-1fd3-42d2-9ecf-17ecfc19d394 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:49.558215573 +0000 UTC m=+37.559082479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls") pod "image-registry-69b67766c6-24kgp" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394") : secret "image-registry-tls" not found
Jul 07 12:42:48.558338 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:48.558254    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls podName:055e0d3d-3f8f-4d6a-acb3-7372795eda3f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:49.558240952 +0000 UTC m=+37.559107852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls") pod "dns-default-c5h8s" (UID: "055e0d3d-3f8f-4d6a-acb3-7372795eda3f") : secret "dns-default-metrics-tls" not found
Jul 07 12:42:48.774074 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:48.773980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gprxp" event={"ID":"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f","Type":"ContainerStarted","Data":"de6668048c62f3defbc77431b8dc3b293c3165984a191de6c9da68183e8d30dd"}
Jul 07 12:42:49.564689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:49.564644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:49.564881 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:49.564738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:49.564881 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:49.564818    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 12:42:49.564881 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:49.564841    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b67766c6-24kgp: secret "image-registry-tls" not found
Jul 07 12:42:49.564881 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:49.564854    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 12:42:49.565053 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:49.564907    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls podName:055e0d3d-3f8f-4d6a-acb3-7372795eda3f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.564891474 +0000 UTC m=+39.565758378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls") pod "dns-default-c5h8s" (UID: "055e0d3d-3f8f-4d6a-acb3-7372795eda3f") : secret "dns-default-metrics-tls" not found
Jul 07 12:42:49.565053 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:49.564921    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls podName:6863a821-1fd3-42d2-9ecf-17ecfc19d394 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.564915054 +0000 UTC m=+39.565781951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls") pod "image-registry-69b67766c6-24kgp" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394") : secret "image-registry-tls" not found
Jul 07 12:42:50.782657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:50.782622    2579 generic.go:356] "Generic (PLEG): container finished" podID="40316917-dd36-48b1-b4ef-acd4c3ab87ab" containerID="ed4633557a12b507ee97e90759f37062793b3d2193f9bb7fc58bdb4b3f9704c8" exitCode=0
Jul 07 12:42:50.783181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:50.782697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerDied","Data":"ed4633557a12b507ee97e90759f37062793b3d2193f9bb7fc58bdb4b3f9704c8"}
Jul 07 12:42:51.343857 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.343823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:51.398404 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.398367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:51.513481 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.513447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:51.576863 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.576835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:51.577062 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.576912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:51.577062 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:51.576993    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 12:42:51.577062 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:51.577007    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 12:42:51.577062 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:51.577019    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b67766c6-24kgp: secret "image-registry-tls" not found
Jul 07 12:42:51.577062 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:51.577064    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls podName:055e0d3d-3f8f-4d6a-acb3-7372795eda3f nodeName:}" failed. No retries permitted until 2026-07-07 12:42:55.577043426 +0000 UTC m=+43.577910337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls") pod "dns-default-c5h8s" (UID: "055e0d3d-3f8f-4d6a-acb3-7372795eda3f") : secret "dns-default-metrics-tls" not found
Jul 07 12:42:51.577321 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:51.577078    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls podName:6863a821-1fd3-42d2-9ecf-17ecfc19d394 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:55.577072201 +0000 UTC m=+43.577939101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls") pod "image-registry-69b67766c6-24kgp" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394") : secret "image-registry-tls" not found
Jul 07 12:42:51.615391 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.615306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:51.788600 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.788549    2579 generic.go:356] "Generic (PLEG): container finished" podID="40316917-dd36-48b1-b4ef-acd4c3ab87ab" containerID="01b49865309cd73817ed8a3c3f584314dec7319c872d2ef84588952b9071cbd7" exitCode=0
Jul 07 12:42:51.788991 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:51.788589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerDied","Data":"01b49865309cd73817ed8a3c3f584314dec7319c872d2ef84588952b9071cbd7"}
Jul 07 12:42:52.793931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:52.793841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" event={"ID":"40316917-dd36-48b1-b4ef-acd4c3ab87ab","Type":"ContainerStarted","Data":"b550519aa8f49cedbdb49b962c2a3259a2452010b1df952eebed177605ccb154"}
Jul 07 12:42:52.795189 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:52.795168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gprxp" event={"ID":"d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f","Type":"ContainerStarted","Data":"5429579ebdfa9c1fc2b2a04647f792873121bfebeb9c6f77ddfba76c787b1b0c"}
Jul 07 12:42:52.817313 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:52.817278    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2h4wj" podStartSLOduration=9.814201927 podStartE2EDuration="40.817265751s" totalImagesPullingTime="31.003063824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:52.816496226 +0000 UTC m=+40.817363142" watchObservedRunningTime="2026-07-07 12:42:52.817265751 +0000 UTC m=+40.818132669"
Jul 07 12:42:52.956559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:52.956529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:53.023763 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:53.023735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:53.085366 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:53.085339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:53.810699 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:53.810664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:53.908066 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:53.908030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:54.700278 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:54.700242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:54.799347 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:54.799312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log" limit=1024
Jul 07 12:42:55.609943 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.609906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:42:55.610337 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.609995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:42:55.610337 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:55.610061    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 12:42:55.610337 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:55.610121    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls podName:055e0d3d-3f8f-4d6a-acb3-7372795eda3f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:03.61010554 +0000 UTC m=+51.610972436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls") pod "dns-default-c5h8s" (UID: "055e0d3d-3f8f-4d6a-acb3-7372795eda3f") : secret "dns-default-metrics-tls" not found
Jul 07 12:42:55.610337 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:55.610130    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 12:42:55.610337 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:55.610146    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b67766c6-24kgp: secret "image-registry-tls" not found
Jul 07 12:42:55.610337 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:55.610201    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls podName:6863a821-1fd3-42d2-9ecf-17ecfc19d394 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:03.61018808 +0000 UTC m=+51.611054977 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls") pod "image-registry-69b67766c6-24kgp" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394") : secret "image-registry-tls" not found
Jul 07 12:42:55.683695 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.683642    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gprxp" podStartSLOduration=4.45066088 podStartE2EDuration="8.683626038s" totalImagesPullingTime="4.232965158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:52.832463701 +0000 UTC m=+40.833330617" watchObservedRunningTime="2026-07-07 12:42:55.683626038 +0000 UTC m=+43.684492958"
Jul 07 12:42:55.684165 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.684149    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"]
Jul 07 12:42:55.715367 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.715342    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"]
Jul 07 12:42:55.715500 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.715432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"
Jul 07 12:42:55.717446 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.717422    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 12:42:55.717681 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.717667    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 12:42:55.722703 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.722679    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-krcml\""
Jul 07 12:42:55.811690 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.811664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q45fn\" (UniqueName: \"kubernetes.io/projected/63227d1c-ede5-4d30-9620-87df207701ad-kube-api-access-q45fn\") pod \"migrator-d57d58cc9-vwj5c\" (UID: \"63227d1c-ede5-4d30-9620-87df207701ad\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"
Jul 07 12:42:55.912697 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.912629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q45fn\" (UniqueName: \"kubernetes.io/projected/63227d1c-ede5-4d30-9620-87df207701ad-kube-api-access-q45fn\") pod \"migrator-d57d58cc9-vwj5c\" (UID: \"63227d1c-ede5-4d30-9620-87df207701ad\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"
Jul 07 12:42:55.922626 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:55.922606    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q45fn\" (UniqueName: \"kubernetes.io/projected/63227d1c-ede5-4d30-9620-87df207701ad-kube-api-access-q45fn\") pod \"migrator-d57d58cc9-vwj5c\" (UID: \"63227d1c-ede5-4d30-9620-87df207701ad\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"
Jul 07 12:42:56.023931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.023895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"
Jul 07 12:42:56.167077 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.167007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c"]
Jul 07 12:42:56.170183 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.170158    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hmpsl"]
Jul 07 12:42:56.170705 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:56.170681    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63227d1c_ede5_4d30_9620_87df207701ad.slice/crio-a39644a82fb62c5c98f0ff2842e4b13a46ac5cc0c8ef6047dc6e89643f468083 WatchSource:0}: Error finding container a39644a82fb62c5c98f0ff2842e4b13a46ac5cc0c8ef6047dc6e89643f468083: Status 404 returned error can't find the container with id a39644a82fb62c5c98f0ff2842e4b13a46ac5cc0c8ef6047dc6e89643f468083
Jul 07 12:42:56.202015 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.201985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hmpsl"]
Jul 07 12:42:56.202147 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.202111    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.204536 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.204492    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 12:42:56.204641 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.204492    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 12:42:56.204695 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.204668    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 12:42:56.204759 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.204728    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 12:42:56.205047 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.205030    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdzl4\""
Jul 07 12:42:56.214559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.214539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.214667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.214564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.214667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.214628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31eee505-35d2-4037-8cf6-4a4263b4ee78-data-volume\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.214769 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.214665    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gxkf\" (UniqueName: \"kubernetes.io/projected/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-api-access-5gxkf\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.214769 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.214736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31eee505-35d2-4037-8cf6-4a4263b4ee78-crio-socket\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.315845 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.315809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31eee505-35d2-4037-8cf6-4a4263b4ee78-crio-socket\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.316026 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.315981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.316026 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.316010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.316128 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.316042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31eee505-35d2-4037-8cf6-4a4263b4ee78-data-volume\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.316128 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.316050    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31eee505-35d2-4037-8cf6-4a4263b4ee78-crio-socket\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.316128 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.316067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gxkf\" (UniqueName: \"kubernetes.io/projected/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-api-access-5gxkf\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.316128 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.316078    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 12:42:56.316269 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.316137    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:56.316269 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.316141    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:56.816125278 +0000 UTC m=+44.816992175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : configmap "kube-rbac-proxy" not found
Jul 07 12:42:56.316269 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.316197    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:56.816181686 +0000 UTC m=+44.817048586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:56.316416 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.316348    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31eee505-35d2-4037-8cf6-4a4263b4ee78-data-volume\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.324768 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.324747    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gxkf\" (UniqueName: \"kubernetes.io/projected/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-api-access-5gxkf\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.806015 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.805976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c" event={"ID":"63227d1c-ede5-4d30-9620-87df207701ad","Type":"ContainerStarted","Data":"a39644a82fb62c5c98f0ff2842e4b13a46ac5cc0c8ef6047dc6e89643f468083"}
Jul 07 12:42:56.819657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.819632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.819811 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:56.819668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:56.819871 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.819810    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:56.819871 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.819857    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:57.819842238 +0000 UTC m=+45.820709135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:56.820091 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.820066    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 12:42:56.820217 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:56.820142    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:57.820125638 +0000 UTC m=+45.820992541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : configmap "kube-rbac-proxy" not found
Jul 07 12:42:57.809853 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:57.809827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c" event={"ID":"63227d1c-ede5-4d30-9620-87df207701ad","Type":"ContainerStarted","Data":"b010972b6bf4b258ddc271931bd87adcc1b86dec6e87668991d265d6540abb36"}
Jul 07 12:42:57.827613 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:57.827589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:57.827691 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:57.827624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:57.827736 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:57.827687    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 12:42:57.827770 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:57.827761    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.82774054 +0000 UTC m=+47.828607443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : configmap "kube-rbac-proxy" not found
Jul 07 12:42:57.827814 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:57.827766    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:57.827814 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:57.827809    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.827796213 +0000 UTC m=+47.828663111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:58.175749 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.175717    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-bh6ll"]
Jul 07 12:42:58.195060 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.195036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-bh6ll"]
Jul 07 12:42:58.195197 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.195123    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.197146 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.197123    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 12:42:58.197146 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.197134    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 12:42:58.197461 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.197444    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 12:42:58.198029 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.198010    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 12:42:58.198102 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.198055    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4rhm9\""
Jul 07 12:42:58.230719 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.230693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-signing-key\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.230863 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.230752    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-signing-cabundle\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.230863 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.230821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbjmv\" (UniqueName: \"kubernetes.io/projected/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-kube-api-access-fbjmv\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.331754 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.331717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbjmv\" (UniqueName: \"kubernetes.io/projected/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-kube-api-access-fbjmv\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.331920 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.331783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-signing-key\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.331920 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.331828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-signing-cabundle\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.332443 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.332426    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-signing-cabundle\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.334128 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.334108    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-signing-key\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.339508 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.339487    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbjmv\" (UniqueName: \"kubernetes.io/projected/0af8b979-e5d5-4bc3-8b6c-44738f42cf41-kube-api-access-fbjmv\") pod \"service-ca-758f599c5-bh6ll\" (UID: \"0af8b979-e5d5-4bc3-8b6c-44738f42cf41\") " pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.504153 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.504070    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-bh6ll"
Jul 07 12:42:58.552963 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.552943    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cs8fp_aebd87aa-c237-4a76-8d8d-80c8072cd0da/dns-node-resolver/0.log"
Jul 07 12:42:58.683299 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.683269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-bh6ll"]
Jul 07 12:42:58.686548 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:42:58.686506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af8b979_e5d5_4bc3_8b6c_44738f42cf41.slice/crio-639191010e43bc008a23285cfd51942d2f850e5a8ce2f67e11ca063ef14fe204 WatchSource:0}: Error finding container 639191010e43bc008a23285cfd51942d2f850e5a8ce2f67e11ca063ef14fe204: Status 404 returned error can't find the container with id 639191010e43bc008a23285cfd51942d2f850e5a8ce2f67e11ca063ef14fe204
Jul 07 12:42:58.813909 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.813781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-bh6ll" event={"ID":"0af8b979-e5d5-4bc3-8b6c-44738f42cf41","Type":"ContainerStarted","Data":"639191010e43bc008a23285cfd51942d2f850e5a8ce2f67e11ca063ef14fe204"}
Jul 07 12:42:58.815444 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.815416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c" event={"ID":"63227d1c-ede5-4d30-9620-87df207701ad","Type":"ContainerStarted","Data":"507b619681979d47ab6d6d156d132feebe2e03e23ff673f24adf50ed85b083f0"}
Jul 07 12:42:58.832472 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:58.832419    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vwj5c" podStartSLOduration=2.328092863 podStartE2EDuration="3.832403733s" totalImagesPullingTime="1.50431087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:58.832016731 +0000 UTC m=+46.832883652" watchObservedRunningTime="2026-07-07 12:42:58.832403733 +0000 UTC m=+46.833270629"
Jul 07 12:42:59.355234 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:59.355206    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-272qj_f4ac942c-b28d-40da-b650-0451e463eb97/node-ca/0.log"
Jul 07 12:42:59.845031 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:59.844986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:59.845499 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:59.845045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:42:59.845499 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:59.845134    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:59.845499 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:42:59.845183    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:03.845168967 +0000 UTC m=+51.846035864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : secret "insights-runtime-extractor-tls" not found
Jul 07 12:42:59.845696 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:42:59.845609    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31eee505-35d2-4037-8cf6-4a4263b4ee78-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:43:00.352679 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:00.352647    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gprxp_d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f/serve-healthcheck-canary/0.log"
Jul 07 12:43:00.552930 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:00.552892    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vwj5c_63227d1c-ede5-4d30-9620-87df207701ad/migrator/0.log"
Jul 07 12:43:00.756343 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:00.756279    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vwj5c_63227d1c-ede5-4d30-9620-87df207701ad/graceful-termination/0.log"
Jul 07 12:43:01.824211 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:01.824177    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-bh6ll" event={"ID":"0af8b979-e5d5-4bc3-8b6c-44738f42cf41","Type":"ContainerStarted","Data":"adb6a0dee636b37bf58d10eded85431965052223d1db29c9ad695f3e5154b543"}
Jul 07 12:43:01.845392 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:01.842613    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-bh6ll" podStartSLOduration=1.3908482850000001 podStartE2EDuration="3.842594886s" totalImagesPullingTime="2.451746601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:01.841835479 +0000 UTC m=+49.842702397" watchObservedRunningTime="2026-07-07 12:43:01.842594886 +0000 UTC m=+49.843461806"
Jul 07 12:43:03.674566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:03.674535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:43:03.674933 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:03.674599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:03.674933 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.674683    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 12:43:03.674933 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.674697    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69b67766c6-24kgp: secret "image-registry-tls" not found
Jul 07 12:43:03.674933 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.674737    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls podName:6863a821-1fd3-42d2-9ecf-17ecfc19d394 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:19.674724659 +0000 UTC m=+67.675591556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls") pod "image-registry-69b67766c6-24kgp" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394") : secret "image-registry-tls" not found
Jul 07 12:43:03.674933 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.674681    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 12:43:03.674933 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.674823    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls podName:055e0d3d-3f8f-4d6a-acb3-7372795eda3f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:19.674806933 +0000 UTC m=+67.675673849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls") pod "dns-default-c5h8s" (UID: "055e0d3d-3f8f-4d6a-acb3-7372795eda3f") : secret "dns-default-metrics-tls" not found
Jul 07 12:43:03.875584 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:03.875547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:43:03.875742 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.875687    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 12:43:03.875786 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:03.875753    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls podName:31eee505-35d2-4037-8cf6-4a4263b4ee78 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:11.875735675 +0000 UTC m=+59.876602589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hmpsl" (UID: "31eee505-35d2-4037-8cf6-4a4263b4ee78") : secret "insights-runtime-extractor-tls" not found
Jul 07 12:43:11.940531 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:11.940488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:43:11.942687 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:11.942665    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31eee505-35d2-4037-8cf6-4a4263b4ee78-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hmpsl\" (UID: \"31eee505-35d2-4037-8cf6-4a4263b4ee78\") " pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:43:12.111327 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:12.111298    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hmpsl"
Jul 07 12:43:12.235977 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:12.235907    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hmpsl"]
Jul 07 12:43:12.239321 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:12.239295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31eee505_35d2_4037_8cf6_4a4263b4ee78.slice/crio-fd330e61e0eb7e79e080fefcbc6642daa8fd64550ba42332f2f38481d9b5a4bf WatchSource:0}: Error finding container fd330e61e0eb7e79e080fefcbc6642daa8fd64550ba42332f2f38481d9b5a4bf: Status 404 returned error can't find the container with id fd330e61e0eb7e79e080fefcbc6642daa8fd64550ba42332f2f38481d9b5a4bf
Jul 07 12:43:12.853510 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:12.853455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmpsl" event={"ID":"31eee505-35d2-4037-8cf6-4a4263b4ee78","Type":"ContainerStarted","Data":"fd330e61e0eb7e79e080fefcbc6642daa8fd64550ba42332f2f38481d9b5a4bf"}
Jul 07 12:43:13.779892 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:13.779861    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mk748"
Jul 07 12:43:16.865381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:16.865346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmpsl" event={"ID":"31eee505-35d2-4037-8cf6-4a4263b4ee78","Type":"ContainerStarted","Data":"54ddec9e86a214abb66ebf1c1140e7489746c40a6b18bf768cb0f0f00d5e27e5"}
Jul 07 12:43:16.865804 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:16.865409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmpsl" event={"ID":"31eee505-35d2-4037-8cf6-4a4263b4ee78","Type":"ContainerStarted","Data":"88a2b147b4fb49e7999a5fad9f2bc119b69d602eda8e2dcf6cfa2a5d2cdafa95"}
Jul 07 12:43:17.182424 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.182385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:43:17.182616 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.182448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:43:17.184767 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.184744    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 12:43:17.184822 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.184744    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 12:43:17.195589 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.195562    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a0476f5-c151-4bec-9b97-adc1b69ea8b5-original-pull-secret\") pod \"global-pull-secret-syncer-9ngf2\" (UID: \"2a0476f5-c151-4bec-9b97-adc1b69ea8b5\") " pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:43:17.195721 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.195699    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1730709-e335-4671-8081-052fcc0072a7-metrics-certs\") pod \"network-metrics-daemon-f6w76\" (UID: \"f1730709-e335-4671-8081-052fcc0072a7\") " pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:43:17.335300 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.335261    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gxjtk\""
Jul 07 12:43:17.336863 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.336830    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9ngf2"
Jul 07 12:43:17.342553 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.342528    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f6w76"
Jul 07 12:43:17.384482 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.384448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:43:17.388131 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.388088    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 12:43:17.398191 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.397460    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 12:43:17.409353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.409295    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjwpr\" (UniqueName: \"kubernetes.io/projected/9f0183ef-0993-4e30-87d9-64c72faee949-kube-api-access-qjwpr\") pod \"network-check-target-zkr6b\" (UID: \"9f0183ef-0993-4e30-87d9-64c72faee949\") " pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:43:17.478865 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.478836    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9ngf2"]
Jul 07 12:43:17.482417 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:17.482391    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0476f5_c151_4bec_9b97_adc1b69ea8b5.slice/crio-770e925a9851003e23d08e1e8c2e1f643342d79bcf3dc17ee71cd47aff3baa1d WatchSource:0}: Error finding container 770e925a9851003e23d08e1e8c2e1f643342d79bcf3dc17ee71cd47aff3baa1d: Status 404 returned error can't find the container with id 770e925a9851003e23d08e1e8c2e1f643342d79bcf3dc17ee71cd47aff3baa1d
Jul 07 12:43:17.495365 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.495317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f6w76"]
Jul 07 12:43:17.499038 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:17.499004    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1730709_e335_4671_8081_052fcc0072a7.slice/crio-98731b5a6f71e758b20b5fb64d6c7d72a982e7cc017ac037967ff513ea974ea8 WatchSource:0}: Error finding container 98731b5a6f71e758b20b5fb64d6c7d72a982e7cc017ac037967ff513ea974ea8: Status 404 returned error can't find the container with id 98731b5a6f71e758b20b5fb64d6c7d72a982e7cc017ac037967ff513ea974ea8
Jul 07 12:43:17.627086 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.627053    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-849mb\""
Jul 07 12:43:17.635886 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.635858    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:43:17.777050 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.777023    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkr6b"]
Jul 07 12:43:17.780345 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:17.780317    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0183ef_0993_4e30_87d9_64c72faee949.slice/crio-92ccb1d0400cd0ec414744f70eebb35c8eec5da2f8b9dffc5a93ccf2a2287c10 WatchSource:0}: Error finding container 92ccb1d0400cd0ec414744f70eebb35c8eec5da2f8b9dffc5a93ccf2a2287c10: Status 404 returned error can't find the container with id 92ccb1d0400cd0ec414744f70eebb35c8eec5da2f8b9dffc5a93ccf2a2287c10
Jul 07 12:43:17.871542 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.871449    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkr6b" event={"ID":"9f0183ef-0993-4e30-87d9-64c72faee949","Type":"ContainerStarted","Data":"92ccb1d0400cd0ec414744f70eebb35c8eec5da2f8b9dffc5a93ccf2a2287c10"}
Jul 07 12:43:17.873224 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.873186    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f6w76" event={"ID":"f1730709-e335-4671-8081-052fcc0072a7","Type":"ContainerStarted","Data":"98731b5a6f71e758b20b5fb64d6c7d72a982e7cc017ac037967ff513ea974ea8"}
Jul 07 12:43:17.874893 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:17.874864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9ngf2" event={"ID":"2a0476f5-c151-4bec-9b97-adc1b69ea8b5","Type":"ContainerStarted","Data":"770e925a9851003e23d08e1e8c2e1f643342d79bcf3dc17ee71cd47aff3baa1d"}
Jul 07 12:43:19.701696 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.701650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:43:19.702175 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.701752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:19.705280 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.705232    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/055e0d3d-3f8f-4d6a-acb3-7372795eda3f-metrics-tls\") pod \"dns-default-c5h8s\" (UID: \"055e0d3d-3f8f-4d6a-acb3-7372795eda3f\") " pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:43:19.706441 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.706394    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"image-registry-69b67766c6-24kgp\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") " pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:19.933903 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.933867    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ncl7q\""
Jul 07 12:43:19.939600 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.939575    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8ss65\""
Jul 07 12:43:19.942101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.942074    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:43:19.948363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:19.948339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:20.948010 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:20.947935    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69b67766c6-24kgp"]
Jul 07 12:43:20.964910 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:20.964878    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"]
Jul 07 12:43:20.997893 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:20.997871    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"]
Jul 07 12:43:20.998053 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:20.998020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.000186 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.000162    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 12:43:21.000353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.000197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 07 12:43:21.000353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.000245    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 12:43:21.000353 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.000245    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 12:43:21.010179 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.010159    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"]
Jul 07 12:43:21.010296 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.010192    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"]
Jul 07 12:43:21.010296 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.010264    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.012363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.012293    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 12:43:21.012363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.012353    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w8pds\""
Jul 07 12:43:21.060691 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.060663    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-bcfsc"]
Jul 07 12:43:21.078868 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.078844    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8949fc898-wcj5r"]
Jul 07 12:43:21.079021 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.078975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:21.081104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.080914    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vth92\""
Jul 07 12:43:21.081104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.080950    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 12:43:21.081104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.081066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 12:43:21.094197 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.094174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-bcfsc"]
Jul 07 12:43:21.094301 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.094231    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8949fc898-wcj5r"]
Jul 07 12:43:21.094301 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.094276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.112150 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.112119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-klusterlet-config\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.112257 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.112152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr9lf\" (UniqueName: \"kubernetes.io/projected/564b55bc-f14a-4d3f-a1de-f28e09c0e648-kube-api-access-qr9lf\") pod \"managed-serviceaccount-addon-agent-79978cc48b-242bm\" (UID: \"564b55bc-f14a-4d3f-a1de-f28e09c0e648\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.112257 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.112192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/564b55bc-f14a-4d3f-a1de-f28e09c0e648-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-79978cc48b-242bm\" (UID: \"564b55bc-f14a-4d3f-a1de-f28e09c0e648\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.112369 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.112277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4k2k\" (UniqueName: \"kubernetes.io/projected/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-kube-api-access-k4k2k\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.112369 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.112329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-tmp\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.213565 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-bound-sa-token\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.213565 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2zpr\" (UniqueName: \"kubernetes.io/projected/b95f988e-bbee-42d0-b2df-3a702689b803-kube-api-access-b2zpr\") pod \"downloads-685bfb5f64-bcfsc\" (UID: \"b95f988e-bbee-42d0-b2df-3a702689b803\") " pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:21.213801 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/564b55bc-f14a-4d3f-a1de-f28e09c0e648-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-79978cc48b-242bm\" (UID: \"564b55bc-f14a-4d3f-a1de-f28e09c0e648\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.213801 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg922\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-kube-api-access-qg922\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.213801 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4k2k\" (UniqueName: \"kubernetes.io/projected/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-kube-api-access-k4k2k\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.213801 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9184802-2a49-4003-afda-e994a598babc-installation-pull-secrets\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.213801 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9184802-2a49-4003-afda-e994a598babc-image-registry-private-configuration\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.213801 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9184802-2a49-4003-afda-e994a598babc-ca-trust-extracted\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.214104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-registry-tls\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.214104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-tmp\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.214104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9184802-2a49-4003-afda-e994a598babc-trusted-ca\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.214104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9184802-2a49-4003-afda-e994a598babc-registry-certificates\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.214104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.213991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-klusterlet-config\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.214104 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.214046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr9lf\" (UniqueName: \"kubernetes.io/projected/564b55bc-f14a-4d3f-a1de-f28e09c0e648-kube-api-access-qr9lf\") pod \"managed-serviceaccount-addon-agent-79978cc48b-242bm\" (UID: \"564b55bc-f14a-4d3f-a1de-f28e09c0e648\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.214310 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.214280    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-tmp\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.216625 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.216597    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/564b55bc-f14a-4d3f-a1de-f28e09c0e648-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-79978cc48b-242bm\" (UID: \"564b55bc-f14a-4d3f-a1de-f28e09c0e648\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.216915 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.216875    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-klusterlet-config\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.232021 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.231994    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4k2k\" (UniqueName: \"kubernetes.io/projected/f7b4b916-1df6-4a9d-a99f-0fed2119cde0-kube-api-access-k4k2k\") pod \"klusterlet-addon-workmgr-656644f584-lwdz8\" (UID: \"f7b4b916-1df6-4a9d-a99f-0fed2119cde0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.234895 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.234873    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr9lf\" (UniqueName: \"kubernetes.io/projected/564b55bc-f14a-4d3f-a1de-f28e09c0e648-kube-api-access-qr9lf\") pod \"managed-serviceaccount-addon-agent-79978cc48b-242bm\" (UID: \"564b55bc-f14a-4d3f-a1de-f28e09c0e648\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.309773 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.309736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:21.315238 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-bound-sa-token\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2zpr\" (UniqueName: \"kubernetes.io/projected/b95f988e-bbee-42d0-b2df-3a702689b803-kube-api-access-b2zpr\") pod \"downloads-685bfb5f64-bcfsc\" (UID: \"b95f988e-bbee-42d0-b2df-3a702689b803\") " pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:21.315381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg922\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-kube-api-access-qg922\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9184802-2a49-4003-afda-e994a598babc-installation-pull-secrets\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9184802-2a49-4003-afda-e994a598babc-image-registry-private-configuration\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315605 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9184802-2a49-4003-afda-e994a598babc-ca-trust-extracted\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315605 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-registry-tls\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315605 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9184802-2a49-4003-afda-e994a598babc-trusted-ca\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.315605 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.315493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9184802-2a49-4003-afda-e994a598babc-registry-certificates\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.317280 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.316363    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9184802-2a49-4003-afda-e994a598babc-registry-certificates\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.317280 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.316415    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9184802-2a49-4003-afda-e994a598babc-ca-trust-extracted\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.317280 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.317237    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9184802-2a49-4003-afda-e994a598babc-trusted-ca\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.318895 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.318869    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9184802-2a49-4003-afda-e994a598babc-image-registry-private-configuration\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.319096 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.319070    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-registry-tls\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.319165 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.319128    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9184802-2a49-4003-afda-e994a598babc-installation-pull-secrets\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.323788 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.323764    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-bound-sa-token\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.324624 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.324605    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg922\" (UniqueName: \"kubernetes.io/projected/b9184802-2a49-4003-afda-e994a598babc-kube-api-access-qg922\") pod \"image-registry-8949fc898-wcj5r\" (UID: \"b9184802-2a49-4003-afda-e994a598babc\") " pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:21.324826 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.324807    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2zpr\" (UniqueName: \"kubernetes.io/projected/b95f988e-bbee-42d0-b2df-3a702689b803-kube-api-access-b2zpr\") pod \"downloads-685bfb5f64-bcfsc\" (UID: \"b95f988e-bbee-42d0-b2df-3a702689b803\") " pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:21.326416 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.326396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"
Jul 07 12:43:21.389036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.389006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:21.403709 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:21.403685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:22.788505 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:22.788480    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"]
Jul 07 12:43:22.791509 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:22.790582    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:22.794693 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:22.794672    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 07 12:43:22.795254 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:22.795231    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s92s4\""
Jul 07 12:43:22.802760 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:22.802467    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"]
Jul 07 12:43:22.933310 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:22.933277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b4193-e750-4055-be6f-307944a5d7c6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m788c\" (UID: \"f85b4193-e750-4055-be6f-307944a5d7c6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:23.014090 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.013655    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c5h8s"]
Jul 07 12:43:23.035438 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.035143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b4193-e750-4055-be6f-307944a5d7c6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m788c\" (UID: \"f85b4193-e750-4055-be6f-307944a5d7c6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:23.035438 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:23.035301    2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 07 12:43:23.035438 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:23.035364    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85b4193-e750-4055-be6f-307944a5d7c6-tls-certificates podName:f85b4193-e750-4055-be6f-307944a5d7c6 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:23.535342352 +0000 UTC m=+71.536209266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f85b4193-e750-4055-be6f-307944a5d7c6-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-m788c" (UID: "f85b4193-e750-4055-be6f-307944a5d7c6") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 07 12:43:23.094006 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.093969    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69b67766c6-24kgp"]
Jul 07 12:43:23.098208 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:23.098176    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6863a821_1fd3_42d2_9ecf_17ecfc19d394.slice/crio-7633ec2ee40556e27cf7feffa40d1326785fecf15eb6b76911a6db1c4341c768 WatchSource:0}: Error finding container 7633ec2ee40556e27cf7feffa40d1326785fecf15eb6b76911a6db1c4341c768: Status 404 returned error can't find the container with id 7633ec2ee40556e27cf7feffa40d1326785fecf15eb6b76911a6db1c4341c768
Jul 07 12:43:23.114414 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.114387    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"]
Jul 07 12:43:23.121935 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:23.121907    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b4b916_1df6_4a9d_a99f_0fed2119cde0.slice/crio-7aab3669f89afce3cf3b95af9576071b66b8e46b9631acaa458e4bed55bbba64 WatchSource:0}: Error finding container 7aab3669f89afce3cf3b95af9576071b66b8e46b9631acaa458e4bed55bbba64: Status 404 returned error can't find the container with id 7aab3669f89afce3cf3b95af9576071b66b8e46b9631acaa458e4bed55bbba64
Jul 07 12:43:23.136188 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.136152    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm"]
Jul 07 12:43:23.137054 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.136994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8949fc898-wcj5r"]
Jul 07 12:43:23.144177 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:23.144150    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9184802_2a49_4003_afda_e994a598babc.slice/crio-9cbebc3fbf562c9a6cf2b6762ffe6b1ea6de5de311a26b823b0835157cc7832d WatchSource:0}: Error finding container 9cbebc3fbf562c9a6cf2b6762ffe6b1ea6de5de311a26b823b0835157cc7832d: Status 404 returned error can't find the container with id 9cbebc3fbf562c9a6cf2b6762ffe6b1ea6de5de311a26b823b0835157cc7832d
Jul 07 12:43:23.177861 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.177832    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-bcfsc"]
Jul 07 12:43:23.185737 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:23.185699    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95f988e_bbee_42d0_b2df_3a702689b803.slice/crio-30569ffc231e64bf83f910033b57087e830cbbafbafd746b39c60f9969846483 WatchSource:0}: Error finding container 30569ffc231e64bf83f910033b57087e830cbbafbafd746b39c60f9969846483: Status 404 returned error can't find the container with id 30569ffc231e64bf83f910033b57087e830cbbafbafd746b39c60f9969846483
Jul 07 12:43:23.539437 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.539360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b4193-e750-4055-be6f-307944a5d7c6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m788c\" (UID: \"f85b4193-e750-4055-be6f-307944a5d7c6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:23.542720 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.542693    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f85b4193-e750-4055-be6f-307944a5d7c6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m788c\" (UID: \"f85b4193-e750-4055-be6f-307944a5d7c6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:23.723116 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.723085    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:23.886572 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.886271    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"]
Jul 07 12:43:23.900546 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:23.900460    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85b4193_e750_4055_be6f_307944a5d7c6.slice/crio-6f49f343b824b2bda57bf76c63c32e26a760f4b3c6ab4ab8ad231418921c68db WatchSource:0}: Error finding container 6f49f343b824b2bda57bf76c63c32e26a760f4b3c6ab4ab8ad231418921c68db: Status 404 returned error can't find the container with id 6f49f343b824b2bda57bf76c63c32e26a760f4b3c6ab4ab8ad231418921c68db
Jul 07 12:43:23.926467 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.925883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9ngf2" event={"ID":"2a0476f5-c151-4bec-9b97-adc1b69ea8b5","Type":"ContainerStarted","Data":"153a663f6b61b2f0af4f59ce740e4576c4f71def04f40aaf4cea8744e4a91174"}
Jul 07 12:43:23.931589 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.930981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8949fc898-wcj5r" event={"ID":"b9184802-2a49-4003-afda-e994a598babc","Type":"ContainerStarted","Data":"e5025de0d237c994085538e0dccee62663204febe4a45e0d995c951338da9de9"}
Jul 07 12:43:23.931589 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.931032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8949fc898-wcj5r" event={"ID":"b9184802-2a49-4003-afda-e994a598babc","Type":"ContainerStarted","Data":"9cbebc3fbf562c9a6cf2b6762ffe6b1ea6de5de311a26b823b0835157cc7832d"}
Jul 07 12:43:23.931589 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.931423    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:23.934899 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.934341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hmpsl" event={"ID":"31eee505-35d2-4037-8cf6-4a4263b4ee78","Type":"ContainerStarted","Data":"802a2c67a8aedcdcb8e157b6329beb68f3517bfefa15b5ad2a52725567e25211"}
Jul 07 12:43:23.938012 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.937414    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkr6b" event={"ID":"9f0183ef-0993-4e30-87d9-64c72faee949","Type":"ContainerStarted","Data":"da51d1d9788b1e72fa52a20e5869413713da79e254382af3ba722bb29b0a2f57"}
Jul 07 12:43:23.938012 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.937967    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:43:23.942218 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.941456    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9ngf2" podStartSLOduration=66.605175774 podStartE2EDuration="1m11.941438976s" totalImagesPullingTime="5.336263202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:23.940065632 +0000 UTC m=+71.940932556" watchObservedRunningTime="2026-07-07 12:43:23.941438976 +0000 UTC m=+71.942305875"
Jul 07 12:43:23.944134 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.944044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f6w76" event={"ID":"f1730709-e335-4671-8081-052fcc0072a7","Type":"ContainerStarted","Data":"c336b421789cc81bda6ed92173b4e019276dcc732e39f18487a4b11e01f1a3df"}
Jul 07 12:43:23.944134 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.944090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f6w76" event={"ID":"f1730709-e335-4671-8081-052fcc0072a7","Type":"ContainerStarted","Data":"6223ab845403505e7189c1dc7727e0f008adcddedcf2c4b39d01233e22a87d8b"}
Jul 07 12:43:23.946108 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.946072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8" event={"ID":"f7b4b916-1df6-4a9d-a99f-0fed2119cde0","Type":"ContainerStarted","Data":"7aab3669f89afce3cf3b95af9576071b66b8e46b9631acaa458e4bed55bbba64"}
Jul 07 12:43:23.949274 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.949009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b67766c6-24kgp" event={"ID":"6863a821-1fd3-42d2-9ecf-17ecfc19d394","Type":"ContainerStarted","Data":"be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086"}
Jul 07 12:43:23.949274 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.949047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b67766c6-24kgp" event={"ID":"6863a821-1fd3-42d2-9ecf-17ecfc19d394","Type":"ContainerStarted","Data":"7633ec2ee40556e27cf7feffa40d1326785fecf15eb6b76911a6db1c4341c768"}
Jul 07 12:43:23.949274 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.949092    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:23.979599 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.974461    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zkr6b" podStartSLOduration=66.937054418 podStartE2EDuration="1m11.974438282s" totalImagesPullingTime="5.037383864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:23.97310893 +0000 UTC m=+71.973975851" watchObservedRunningTime="2026-07-07 12:43:23.974438282 +0000 UTC m=+71.975305204"
Jul 07 12:43:23.986689 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.986597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c5h8s" event={"ID":"055e0d3d-3f8f-4d6a-acb3-7372795eda3f","Type":"ContainerStarted","Data":"c0dc91dafa07dbdcdd73c78b441b55f8fa526f6ba9d307da88532967a9783df1"}
Jul 07 12:43:23.989142 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.989072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c" event={"ID":"f85b4193-e750-4055-be6f-307944a5d7c6","Type":"ContainerStarted","Data":"6f49f343b824b2bda57bf76c63c32e26a760f4b3c6ab4ab8ad231418921c68db"}
Jul 07 12:43:23.991348 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.991288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-bcfsc" event={"ID":"b95f988e-bbee-42d0-b2df-3a702689b803","Type":"ContainerStarted","Data":"30569ffc231e64bf83f910033b57087e830cbbafbafd746b39c60f9969846483"}
Jul 07 12:43:23.993826 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:23.993767    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm" event={"ID":"564b55bc-f14a-4d3f-a1de-f28e09c0e648","Type":"ContainerStarted","Data":"6e068337eb8100349be0f350c38518b675ff2891ce361793375b96f2e2b9e76a"}
Jul 07 12:43:24.026538 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:24.025766    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8949fc898-wcj5r" podStartSLOduration=3.025748657 podStartE2EDuration="3.025748657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:24.025025605 +0000 UTC m=+72.025892525" watchObservedRunningTime="2026-07-07 12:43:24.025748657 +0000 UTC m=+72.026615571"
Jul 07 12:43:24.026538 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:24.026017    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hmpsl" podStartSLOduration=17.580619526 podStartE2EDuration="28.02600444s" totalImagesPullingTime="10.445384914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:24.008172761 +0000 UTC m=+72.009039681" watchObservedRunningTime="2026-07-07 12:43:24.02600444 +0000 UTC m=+72.026871356"
Jul 07 12:43:24.042873 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:24.042136    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69b67766c6-24kgp" podStartSLOduration=47.042118645 podStartE2EDuration="47.042118645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:24.041822202 +0000 UTC m=+72.042689123" watchObservedRunningTime="2026-07-07 12:43:24.042118645 +0000 UTC m=+72.042985564"
Jul 07 12:43:24.065635 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:24.065575    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f6w76" podStartSLOduration=66.763948795 podStartE2EDuration="1m12.065554928s" totalImagesPullingTime="5.301606133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:24.06420268 +0000 UTC m=+72.065069612" watchObservedRunningTime="2026-07-07 12:43:24.065554928 +0000 UTC m=+72.066421844"
Jul 07 12:43:29.019743 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.019702    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c5h8s" event={"ID":"055e0d3d-3f8f-4d6a-acb3-7372795eda3f","Type":"ContainerStarted","Data":"76cc198cfb65ef9448dd462fc6689793c6fbe1ad49b942759a00a92aecf647ae"}
Jul 07 12:43:29.020229 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.019796    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c5h8s" event={"ID":"055e0d3d-3f8f-4d6a-acb3-7372795eda3f","Type":"ContainerStarted","Data":"2304b0755ac2e3c08108337529d4f350b52a5a04879733f5946dbb2fbfb5cdd3"}
Jul 07 12:43:29.020229 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.019825    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:43:29.021534 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.021497    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c" event={"ID":"f85b4193-e750-4055-be6f-307944a5d7c6","Type":"ContainerStarted","Data":"e93bd3c377ee1813367b508a4dedf23d1b5365c5ee21b5956f78a361884438a6"}
Jul 07 12:43:29.021744 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.021716    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:29.023715 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.023365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm" event={"ID":"564b55bc-f14a-4d3f-a1de-f28e09c0e648","Type":"ContainerStarted","Data":"5ab6e9d61e39a9518ea1805737cc63caeb85fc42a3507315d2da1233dc9f0e8e"}
Jul 07 12:43:29.025182 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.025151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8" event={"ID":"f7b4b916-1df6-4a9d-a99f-0fed2119cde0","Type":"ContainerStarted","Data":"b748f7676721a5f72af45231da26ed70c606b54f81958bb9b33023526fdd28b4"}
Jul 07 12:43:29.025378 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.025338    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:29.027344 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.027324    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c"
Jul 07 12:43:29.027424 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.027385    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8"
Jul 07 12:43:29.037260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.037214    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c5h8s" podStartSLOduration=36.519021899 podStartE2EDuration="42.037197604s" totalImagesPullingTime="5.518175705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:29.035928223 +0000 UTC m=+77.036795143" watchObservedRunningTime="2026-07-07 12:43:29.037197604 +0000 UTC m=+77.038064523"
Jul 07 12:43:29.053056 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.052995    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656644f584-lwdz8" podStartSLOduration=3.60386374 podStartE2EDuration="9.052973624s" totalImagesPullingTime="5.449109884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:29.051456815 +0000 UTC m=+77.052323738" watchObservedRunningTime="2026-07-07 12:43:29.052973624 +0000 UTC m=+77.053840544"
Jul 07 12:43:29.065823 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.065763    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79978cc48b-242bm" podStartSLOduration=3.6506679269999998 podStartE2EDuration="9.065747844s" totalImagesPullingTime="5.415079917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:29.065071892 +0000 UTC m=+77.065938825" watchObservedRunningTime="2026-07-07 12:43:29.065747844 +0000 UTC m=+77.066614763"
Jul 07 12:43:29.079094 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:29.079056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m788c" podStartSLOduration=2.4350141 podStartE2EDuration="7.079047069s" totalImagesPullingTime="4.644032969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:29.079041 +0000 UTC m=+77.079907920" watchObservedRunningTime="2026-07-07 12:43:29.079047069 +0000 UTC m=+77.079913985"
Jul 07 12:43:38.249313 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.249189    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bpsxz"]
Jul 07 12:43:38.265023 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.264995    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.267621 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.267576    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 12:43:38.267753 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.267724    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 12:43:38.267813 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.267802    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w4kcv\""
Jul 07 12:43:38.268388 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.268095    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 12:43:38.268388 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.268229    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 12:43:38.268388 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.268279    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 12:43:38.268646 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.268460    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 12:43:38.361929 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.361899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8268t\" (UniqueName: \"kubernetes.io/projected/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-kube-api-access-8268t\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362100 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.361936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-root\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362100 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.361968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-metrics-client-ca\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362100 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.362058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-textfile\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.362117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.362148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.362170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-sys\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.362200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-wtmp\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.362260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.362247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-tls\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.462942 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.462874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-tls\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.462942 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.462930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8268t\" (UniqueName: \"kubernetes.io/projected/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-kube-api-access-8268t\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.462963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-root\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.462999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-metrics-client-ca\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-textfile\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463163 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-sys\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463448 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-wtmp\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463448 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463336    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-wtmp\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.463568 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:38.463446    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 12:43:38.463568 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:38.463503    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-tls podName:cddd83b3-c896-4ded-81b6-7ea26d8bbf2f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:38.963483005 +0000 UTC m=+86.964349903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-tls") pod "node-exporter-bpsxz" (UID: "cddd83b3-c896-4ded-81b6-7ea26d8bbf2f") : secret "node-exporter-tls" not found
Jul 07 12:43:38.463882 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.463858    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-root\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.464463 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.464190    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-sys\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.464592 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.464466    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-metrics-client-ca\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.464850 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.464829    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-textfile\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.465145 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.465074    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.473245 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.473223    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8268t\" (UniqueName: \"kubernetes.io/projected/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-kube-api-access-8268t\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.474549 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.474509    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.967157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.967116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-tls\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:38.969989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:38.969959    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cddd83b3-c896-4ded-81b6-7ea26d8bbf2f-node-exporter-tls\") pod \"node-exporter-bpsxz\" (UID: \"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f\") " pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:39.031807 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:39.031777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c5h8s"
Jul 07 12:43:39.176649 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:39.175970    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bpsxz"
Jul 07 12:43:41.409194 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:41.409151    2579 patch_prober.go:28] interesting pod/image-registry-8949fc898-wcj5r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 12:43:41.409802 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:41.409212    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8949fc898-wcj5r" podUID="b9184802-2a49-4003-afda-e994a598babc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 12:43:42.290857 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:42.290829    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcddd83b3_c896_4ded_81b6_7ea26d8bbf2f.slice/crio-3671ebbecce3e13c683adfcc5f218cf882bd554edac1aa637ee1fbc15ecc936b WatchSource:0}: Error finding container 3671ebbecce3e13c683adfcc5f218cf882bd554edac1aa637ee1fbc15ecc936b: Status 404 returned error can't find the container with id 3671ebbecce3e13c683adfcc5f218cf882bd554edac1aa637ee1fbc15ecc936b
Jul 07 12:43:43.091637 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.091600    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-bcfsc" event={"ID":"b95f988e-bbee-42d0-b2df-3a702689b803","Type":"ContainerStarted","Data":"afcb67b6b7c51a084688aaedae1478daa22de9fe93b076696db1819309dfe636"}
Jul 07 12:43:43.092383 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.092167    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:43.093362 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.093333    2579 patch_prober.go:28] interesting pod/downloads-685bfb5f64-bcfsc container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.14:8080/\": dial tcp 10.133.0.14:8080: connect: connection refused" start-of-body=
Jul 07 12:43:43.093450 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.093385    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-bcfsc" podUID="b95f988e-bbee-42d0-b2df-3a702689b803" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.14:8080/\": dial tcp 10.133.0.14:8080: connect: connection refused"
Jul 07 12:43:43.093991 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.093964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bpsxz" event={"ID":"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f","Type":"ContainerStarted","Data":"3671ebbecce3e13c683adfcc5f218cf882bd554edac1aa637ee1fbc15ecc936b"}
Jul 07 12:43:43.109473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.109426    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-bcfsc" podStartSLOduration=2.605964981 podStartE2EDuration="22.109410002s" totalImagesPullingTime="19.503445021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:43.108488028 +0000 UTC m=+91.109354949" watchObservedRunningTime="2026-07-07 12:43:43.109410002 +0000 UTC m=+91.110276922"
Jul 07 12:43:43.955953 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:43.955924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:44.100005 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:44.099964    2579 generic.go:356] "Generic (PLEG): container finished" podID="cddd83b3-c896-4ded-81b6-7ea26d8bbf2f" containerID="f38adeebf6d19553118cc22d8c9736be06594e71e836f9afec492d2b94dead12" exitCode=0
Jul 07 12:43:44.100497 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:44.100128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bpsxz" event={"ID":"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f","Type":"ContainerDied","Data":"f38adeebf6d19553118cc22d8c9736be06594e71e836f9afec492d2b94dead12"}
Jul 07 12:43:44.113263 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:44.113236    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-bcfsc"
Jul 07 12:43:45.007632 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.007600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8949fc898-wcj5r"
Jul 07 12:43:45.106757 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.106712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bpsxz" event={"ID":"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f","Type":"ContainerStarted","Data":"239f18367a788502dc5cfafcbcb9a1e4f34f9bfae695137c476f00ce8c096ac9"}
Jul 07 12:43:45.107207 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.106784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bpsxz" event={"ID":"cddd83b3-c896-4ded-81b6-7ea26d8bbf2f","Type":"ContainerStarted","Data":"f1be7dcef7010cd9f02f97af873c7441850993d7c246b289f04e4d51d815a691"}
Jul 07 12:43:45.169248 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.169186    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bpsxz" podStartSLOduration=6.025422352 podStartE2EDuration="7.169155041s" totalImagesPullingTime="1.061205431s" totalInitContainerRuntime="82.527258ms" isStatefulPod=true podCreationTimestamp="2026-07-07 12:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:45.169113204 +0000 UTC m=+93.169980124" watchObservedRunningTime="2026-07-07 12:43:45.169155041 +0000 UTC m=+93.170021950"
Jul 07 12:43:45.685198 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.685163    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bd6fdd76c-8klhv"]
Jul 07 12:43:45.711206 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.711167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bd6fdd76c-8klhv"]
Jul 07 12:43:45.711379 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.711280    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.713344 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.713319    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 12:43:45.713478 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.713416    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8vd9j\""
Jul 07 12:43:45.713579 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.713481    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 12:43:45.713691 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.713579    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 12:43:45.713754 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.713717    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 12:43:45.713867 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.713834    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 12:43:45.720220 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.720203    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 12:43:45.821358 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bf84\" (UniqueName: \"kubernetes.io/projected/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-kube-api-access-2bf84\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.821569 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-service-ca\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.821569 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821442    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-serving-cert\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.821569 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-oauth-config\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.821569 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-trusted-ca-bundle\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.821759 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-config\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.821759 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.821622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-oauth-serving-cert\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.922863 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.922826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bf84\" (UniqueName: \"kubernetes.io/projected/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-kube-api-access-2bf84\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.922885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-service-ca\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.922916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-serving-cert\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.922942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-oauth-config\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.922969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-trusted-ca-bundle\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.923019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-config\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923292 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.923043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-oauth-serving-cert\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.923953 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.923910    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-service-ca\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.924630 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.924603    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-trusted-ca-bundle\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.924798 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.924779    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-config\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.924874 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.924836    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-oauth-serving-cert\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.926080 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.926057    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-oauth-config\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.926176 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.926079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-serving-cert\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:45.942352 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:45.942288    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bf84\" (UniqueName: \"kubernetes.io/projected/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-kube-api-access-2bf84\") pod \"console-7bd6fdd76c-8klhv\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") " pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:46.023036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:46.022997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:46.199882 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:46.199807    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bd6fdd76c-8klhv"]
Jul 07 12:43:46.203538 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:43:46.203498    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d26da1_a7ec_40d3_b86f_8f6eb443b427.slice/crio-c550b5c3fceeb2f61a162daea045f599862d528866dee5e06e4b2e57bc77a8bc WatchSource:0}: Error finding container c550b5c3fceeb2f61a162daea045f599862d528866dee5e06e4b2e57bc77a8bc: Status 404 returned error can't find the container with id c550b5c3fceeb2f61a162daea045f599862d528866dee5e06e4b2e57bc77a8bc
Jul 07 12:43:47.118141 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:47.118096    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd6fdd76c-8klhv" event={"ID":"a8d26da1-a7ec-40d3-b86f-8f6eb443b427","Type":"ContainerStarted","Data":"c550b5c3fceeb2f61a162daea045f599862d528866dee5e06e4b2e57bc77a8bc"}
Jul 07 12:43:49.010042 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.009980    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69b67766c6-24kgp" podUID="6863a821-1fd3-42d2-9ecf-17ecfc19d394" containerName="registry" containerID="cri-o://be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086" gracePeriod=30
Jul 07 12:43:49.626059 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.626030    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:49.756495 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756421    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756495 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756480    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-bound-sa-token\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756548    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-certificates\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756583    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-image-registry-private-configuration\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756609    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-installation-pull-secrets\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756650    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zqwg\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-kube-api-access-7zqwg\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756682    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6863a821-1fd3-42d2-9ecf-17ecfc19d394-ca-trust-extracted\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.756734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756724    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-trusted-ca\") pod \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\" (UID: \"6863a821-1fd3-42d2-9ecf-17ecfc19d394\") "
Jul 07 12:43:49.757028 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.756898    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:43:49.757360 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.757247    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:43:49.760070 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.760027    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:43:49.760170 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.760104    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:43:49.760381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.760354    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:43:49.760481 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.760421    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:43:49.760680 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.760648    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-kube-api-access-7zqwg" (OuterVolumeSpecName: "kube-api-access-7zqwg") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "kube-api-access-7zqwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:43:49.770819 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.770792    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6863a821-1fd3-42d2-9ecf-17ecfc19d394-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6863a821-1fd3-42d2-9ecf-17ecfc19d394" (UID: "6863a821-1fd3-42d2-9ecf-17ecfc19d394"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:43:49.858037 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858009    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-trusted-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858037 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858038    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858047    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-bound-sa-token\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858055    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6863a821-1fd3-42d2-9ecf-17ecfc19d394-registry-certificates\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858070    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-image-registry-private-configuration\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858083    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6863a821-1fd3-42d2-9ecf-17ecfc19d394-installation-pull-secrets\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858096    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zqwg\" (UniqueName: \"kubernetes.io/projected/6863a821-1fd3-42d2-9ecf-17ecfc19d394-kube-api-access-7zqwg\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:49.858181 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:49.858110    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6863a821-1fd3-42d2-9ecf-17ecfc19d394-ca-trust-extracted\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:43:50.133328 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.133286    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd6fdd76c-8klhv" event={"ID":"a8d26da1-a7ec-40d3-b86f-8f6eb443b427","Type":"ContainerStarted","Data":"3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883"}
Jul 07 12:43:50.135126 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.135099    2579 generic.go:356] "Generic (PLEG): container finished" podID="6863a821-1fd3-42d2-9ecf-17ecfc19d394" containerID="be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086" exitCode=0
Jul 07 12:43:50.135242 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.135133    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69b67766c6-24kgp"
Jul 07 12:43:50.135242 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.135147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b67766c6-24kgp" event={"ID":"6863a821-1fd3-42d2-9ecf-17ecfc19d394","Type":"ContainerDied","Data":"be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086"}
Jul 07 12:43:50.135242 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.135195    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69b67766c6-24kgp" event={"ID":"6863a821-1fd3-42d2-9ecf-17ecfc19d394","Type":"ContainerDied","Data":"7633ec2ee40556e27cf7feffa40d1326785fecf15eb6b76911a6db1c4341c768"}
Jul 07 12:43:50.135242 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.135220    2579 scope.go:117] "RemoveContainer" containerID="be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086"
Jul 07 12:43:50.149277 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.149236    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bd6fdd76c-8klhv" podStartSLOduration=1.478306914 podStartE2EDuration="5.149219979s" totalImagesPullingTime="3.670913065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:50.147886249 +0000 UTC m=+98.148753168" watchObservedRunningTime="2026-07-07 12:43:50.149219979 +0000 UTC m=+98.150087297"
Jul 07 12:43:50.154186 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.154163    2579 scope.go:117] "RemoveContainer" containerID="be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086"
Jul 07 12:43:50.154491 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:43:50.154465    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086\": container with ID starting with be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086 not found: ID does not exist" containerID="be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086"
Jul 07 12:43:50.154604 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.154502    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086"} err="failed to get container status \"be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086\": rpc error: code = NotFound desc = could not find container \"be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086\": container with ID starting with be595e914af45b3bbda5d6174d69367298867ad84a6516c1fcc3611a74176086 not found: ID does not exist"
Jul 07 12:43:50.159625 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.159601    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69b67766c6-24kgp"]
Jul 07 12:43:50.163004 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.162984    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69b67766c6-24kgp"]
Jul 07 12:43:50.517838 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:50.517761    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6863a821-1fd3-42d2-9ecf-17ecfc19d394" path="/var/lib/kubelet/pods/6863a821-1fd3-42d2-9ecf-17ecfc19d394/volumes"
Jul 07 12:43:56.007816 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:56.007784    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zkr6b"
Jul 07 12:43:56.023290 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:56.023255    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:56.023423 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:56.023312    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:56.027675 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:56.027655    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:43:56.162256 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:43:56.162229    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:45:07.059941 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.059857    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f6bcf6c44-wfrtc"]
Jul 07 12:45:07.060358 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.060145    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6863a821-1fd3-42d2-9ecf-17ecfc19d394" containerName="registry"
Jul 07 12:45:07.060358 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.060159    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6863a821-1fd3-42d2-9ecf-17ecfc19d394" containerName="registry"
Jul 07 12:45:07.060358 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.060211    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6863a821-1fd3-42d2-9ecf-17ecfc19d394" containerName="registry"
Jul 07 12:45:07.062006 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.061985    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.072824 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.072800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f6bcf6c44-wfrtc"]
Jul 07 12:45:07.191046 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-service-ca\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.191170 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-trusted-ca-bundle\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.191170 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz442\" (UniqueName: \"kubernetes.io/projected/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-kube-api-access-kz442\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.191170 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-serving-cert\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.191170 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191147    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-oauth-config\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.191304 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-oauth-serving-cert\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.191304 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.191204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-config\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.291961 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.291932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-serving-cert\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292119 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.291969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-oauth-config\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292119 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.291986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-oauth-serving-cert\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292119 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-config\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292119 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-service-ca\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292119 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-trusted-ca-bundle\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292358 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz442\" (UniqueName: \"kubernetes.io/projected/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-kube-api-access-kz442\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292856 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292828    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-config\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.292856 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292849    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-oauth-serving-cert\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.293030 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.292932    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-service-ca\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.293101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.293075    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-trusted-ca-bundle\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.294536 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.294502    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-oauth-config\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.294621 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.294558    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-serving-cert\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.300652 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.300628    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz442\" (UniqueName: \"kubernetes.io/projected/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-kube-api-access-kz442\") pod \"console-6f6bcf6c44-wfrtc\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") " pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.371058 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.371037    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:07.514941 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:07.514909    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f6bcf6c44-wfrtc"]
Jul 07 12:45:07.520714 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:45:07.520691    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1b7169_eb85_4b29_8dcd_ddcd90e87a92.slice/crio-94408e7b6c6ba5ae5bf73ed01d499bf04d3fb94659ad5d78a674f54a8c226583 WatchSource:0}: Error finding container 94408e7b6c6ba5ae5bf73ed01d499bf04d3fb94659ad5d78a674f54a8c226583: Status 404 returned error can't find the container with id 94408e7b6c6ba5ae5bf73ed01d499bf04d3fb94659ad5d78a674f54a8c226583
Jul 07 12:45:08.413406 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:08.413366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f6bcf6c44-wfrtc" event={"ID":"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92","Type":"ContainerStarted","Data":"b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674"}
Jul 07 12:45:08.413406 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:08.413414    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f6bcf6c44-wfrtc" event={"ID":"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92","Type":"ContainerStarted","Data":"94408e7b6c6ba5ae5bf73ed01d499bf04d3fb94659ad5d78a674f54a8c226583"}
Jul 07 12:45:08.439530 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:08.439481    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f6bcf6c44-wfrtc" podStartSLOduration=1.439467512 podStartE2EDuration="1.439467512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:45:08.439395405 +0000 UTC m=+176.440262323" watchObservedRunningTime="2026-07-07 12:45:08.439467512 +0000 UTC m=+176.440334431"
Jul 07 12:45:17.371151 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:17.371109    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:17.371672 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:17.371173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:17.375705 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:17.375682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:17.447995 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:17.447971    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:45:17.493488 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:17.493462    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bd6fdd76c-8klhv"]
Jul 07 12:45:42.513389 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.513332    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7bd6fdd76c-8klhv" podUID="a8d26da1-a7ec-40d3-b86f-8f6eb443b427" containerName="console" containerID="cri-o://3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883" gracePeriod=15
Jul 07 12:45:42.749340 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.749318    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:45:42.852986 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.852956    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-serving-cert\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.852994    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-config\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-oauth-serving-cert\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853083    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-oauth-config\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853119    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bf84\" (UniqueName: \"kubernetes.io/projected/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-kube-api-access-2bf84\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853139    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-service-ca\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-trusted-ca-bundle\") pod \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\" (UID: \"a8d26da1-a7ec-40d3-b86f-8f6eb443b427\") "
Jul 07 12:45:42.853595 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853498    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:45:42.853697 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853507    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-config" (OuterVolumeSpecName: "console-config") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:45:42.853697 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853678    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:45:42.853787 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.853704    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:45:42.855551 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.855495    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:45:42.855716 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.855680    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:45:42.855776 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.855721    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-kube-api-access-2bf84" (OuterVolumeSpecName: "kube-api-access-2bf84") pod "a8d26da1-a7ec-40d3-b86f-8f6eb443b427" (UID: "a8d26da1-a7ec-40d3-b86f-8f6eb443b427"). InnerVolumeSpecName "kube-api-access-2bf84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:45:42.953820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953785    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-oauth-serving-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:42.953820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953815    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-oauth-config\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:42.953820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953825    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bf84\" (UniqueName: \"kubernetes.io/projected/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-kube-api-access-2bf84\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:42.953820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953834    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:42.954073 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953842    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-trusted-ca-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:42.954073 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953851    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-serving-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:42.954073 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:42.953859    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8d26da1-a7ec-40d3-b86f-8f6eb443b427-console-config\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:45:43.532155 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.532125    2579 generic.go:356] "Generic (PLEG): container finished" podID="a8d26da1-a7ec-40d3-b86f-8f6eb443b427" containerID="3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883" exitCode=0
Jul 07 12:45:43.532559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.532168    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd6fdd76c-8klhv"
Jul 07 12:45:43.532559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.532192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd6fdd76c-8klhv" event={"ID":"a8d26da1-a7ec-40d3-b86f-8f6eb443b427","Type":"ContainerDied","Data":"3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883"}
Jul 07 12:45:43.532559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.532229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd6fdd76c-8klhv" event={"ID":"a8d26da1-a7ec-40d3-b86f-8f6eb443b427","Type":"ContainerDied","Data":"c550b5c3fceeb2f61a162daea045f599862d528866dee5e06e4b2e57bc77a8bc"}
Jul 07 12:45:43.532559 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.532248    2579 scope.go:117] "RemoveContainer" containerID="3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883"
Jul 07 12:45:43.540712 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.540694    2579 scope.go:117] "RemoveContainer" containerID="3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883"
Jul 07 12:45:43.540966 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:45:43.540948    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883\": container with ID starting with 3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883 not found: ID does not exist" containerID="3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883"
Jul 07 12:45:43.541028 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.540975    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883"} err="failed to get container status \"3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883\": rpc error: code = NotFound desc = could not find container \"3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883\": container with ID starting with 3f67ec68ba2a887c2816ab6c013987bded5e11071060ea5d279f84ecfacf9883 not found: ID does not exist"
Jul 07 12:45:43.554602 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.554579    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bd6fdd76c-8klhv"]
Jul 07 12:45:43.558556 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:43.558536    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bd6fdd76c-8klhv"]
Jul 07 12:45:44.516776 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:45:44.516746    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d26da1-a7ec-40d3-b86f-8f6eb443b427" path="/var/lib/kubelet/pods/a8d26da1-a7ec-40d3-b86f-8f6eb443b427/volumes"
Jul 07 12:46:19.194031 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.193998    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bc4896648-w4n64"]
Jul 07 12:46:19.194464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.194253    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d26da1-a7ec-40d3-b86f-8f6eb443b427" containerName="console"
Jul 07 12:46:19.194464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.194263    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d26da1-a7ec-40d3-b86f-8f6eb443b427" containerName="console"
Jul 07 12:46:19.194464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.194311    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d26da1-a7ec-40d3-b86f-8f6eb443b427" containerName="console"
Jul 07 12:46:19.196860 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.196842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.212091 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.212061    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc4896648-w4n64"]
Jul 07 12:46:19.295426 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-trusted-ca-bundle\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.295639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-config\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.295639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-oauth-serving-cert\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.295639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckwgw\" (UniqueName: \"kubernetes.io/projected/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-kube-api-access-ckwgw\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.295639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-service-ca\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.295639 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-oauth-config\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.295857 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.295699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-serving-cert\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.396809 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.396765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-oauth-serving-cert\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.396809 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.396809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckwgw\" (UniqueName: \"kubernetes.io/projected/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-kube-api-access-ckwgw\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397059 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.396834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-service-ca\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397059 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.396858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-oauth-config\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397059 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-serving-cert\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397216 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-trusted-ca-bundle\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397216 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-config\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397511 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397475    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-service-ca\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397656 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397639    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-oauth-serving-cert\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397753 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397736    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-config\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.397930 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.397903    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-trusted-ca-bundle\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.399499 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.399477    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-oauth-config\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.399607 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.399473    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-serving-cert\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.405667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.405639    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckwgw\" (UniqueName: \"kubernetes.io/projected/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-kube-api-access-ckwgw\") pod \"console-7bc4896648-w4n64\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") " pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.505875 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.505791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:19.637032 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.636997    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc4896648-w4n64"]
Jul 07 12:46:19.640614 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:46:19.640590    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b2d5dc_b5d8_4205_98bf_be53bf98f58f.slice/crio-206b2905be5d4977cb2844d181cbc5f375a764034ba2b5c2c20ac9ae18a224e7 WatchSource:0}: Error finding container 206b2905be5d4977cb2844d181cbc5f375a764034ba2b5c2c20ac9ae18a224e7: Status 404 returned error can't find the container with id 206b2905be5d4977cb2844d181cbc5f375a764034ba2b5c2c20ac9ae18a224e7
Jul 07 12:46:19.650202 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:19.650173    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc4896648-w4n64" event={"ID":"68b2d5dc-b5d8-4205-98bf-be53bf98f58f","Type":"ContainerStarted","Data":"206b2905be5d4977cb2844d181cbc5f375a764034ba2b5c2c20ac9ae18a224e7"}
Jul 07 12:46:20.655499 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:20.655462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc4896648-w4n64" event={"ID":"68b2d5dc-b5d8-4205-98bf-be53bf98f58f","Type":"ContainerStarted","Data":"7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39"}
Jul 07 12:46:20.672564 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:20.672489    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bc4896648-w4n64" podStartSLOduration=1.67247327 podStartE2EDuration="1.67247327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:46:20.671111598 +0000 UTC m=+248.671978521" watchObservedRunningTime="2026-07-07 12:46:20.67247327 +0000 UTC m=+248.673340188"
Jul 07 12:46:29.506010 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:29.505972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:29.506564 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:29.506036    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:29.510815 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:29.510795    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:29.691130 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:29.691101    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:46:29.737769 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:29.737733    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f6bcf6c44-wfrtc"]
Jul 07 12:46:54.757583 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:54.757549    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6f6bcf6c44-wfrtc" podUID="fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" containerName="console" containerID="cri-o://b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674" gracePeriod=15
Jul 07 12:46:54.991561 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:54.991536    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:46:55.064577 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064549    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-service-ca\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.064747 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064590    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-serving-cert\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.064747 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064616    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz442\" (UniqueName: \"kubernetes.io/projected/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-kube-api-access-kz442\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.064747 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064651    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-config\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.064747 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064672    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-oauth-serving-cert\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.064747 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-trusted-ca-bundle\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.064747 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.064728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-oauth-config\") pod \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\" (UID: \"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92\") "
Jul 07 12:46:55.065077 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.065050    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:46:55.065200 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.065170    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:46:55.065257 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.065181    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-config" (OuterVolumeSpecName: "console-config") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:46:55.065257 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.065219    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:46:55.066740 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.066710    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:46:55.066830 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.066804    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:46:55.066906 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.066891    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-kube-api-access-kz442" (OuterVolumeSpecName: "kube-api-access-kz442") pod "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" (UID: "fc1b7169-eb85-4b29-8dcd-ddcd90e87a92"). InnerVolumeSpecName "kube-api-access-kz442". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:46:55.165514 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165482    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-oauth-serving-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.165514 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165507    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-trusted-ca-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.165514 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165534    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-oauth-config\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.165514 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165544    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.165774 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165553    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-serving-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.165774 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165563    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz442\" (UniqueName: \"kubernetes.io/projected/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-kube-api-access-kz442\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.165774 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.165572    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92-console-config\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:46:55.777337 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.777303    2579 generic.go:356] "Generic (PLEG): container finished" podID="fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" containerID="b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674" exitCode=0
Jul 07 12:46:55.777822 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.777352    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f6bcf6c44-wfrtc"
Jul 07 12:46:55.777822 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.777390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f6bcf6c44-wfrtc" event={"ID":"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92","Type":"ContainerDied","Data":"b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674"}
Jul 07 12:46:55.777822 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.777437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f6bcf6c44-wfrtc" event={"ID":"fc1b7169-eb85-4b29-8dcd-ddcd90e87a92","Type":"ContainerDied","Data":"94408e7b6c6ba5ae5bf73ed01d499bf04d3fb94659ad5d78a674f54a8c226583"}
Jul 07 12:46:55.777822 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.777458    2579 scope.go:117] "RemoveContainer" containerID="b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674"
Jul 07 12:46:55.785630 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.785616    2579 scope.go:117] "RemoveContainer" containerID="b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674"
Jul 07 12:46:55.785853 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:46:55.785834    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674\": container with ID starting with b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674 not found: ID does not exist" containerID="b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674"
Jul 07 12:46:55.785902 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.785861    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674"} err="failed to get container status \"b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674\": rpc error: code = NotFound desc = could not find container \"b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674\": container with ID starting with b504b0d1b94f8781401571de6aa9424aa5e16a7564369def73500e6fce792674 not found: ID does not exist"
Jul 07 12:46:55.796532 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.796492    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f6bcf6c44-wfrtc"]
Jul 07 12:46:55.801558 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:55.801538    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f6bcf6c44-wfrtc"]
Jul 07 12:46:56.516933 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:46:56.516904    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" path="/var/lib/kubelet/pods/fc1b7169-eb85-4b29-8dcd-ddcd90e87a92/volumes"
Jul 07 12:47:12.376625 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:12.376593    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 12:47:17.207535 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.207487    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"]
Jul 07 12:47:17.209868 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.207757    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" containerName="console"
Jul 07 12:47:17.209868 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.207768    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" containerName="console"
Jul 07 12:47:17.209868 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.207816    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc1b7169-eb85-4b29-8dcd-ddcd90e87a92" containerName="console"
Jul 07 12:47:17.210723 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.210707    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.213843 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.213823    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 12:47:17.213985 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.213964    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjgf7\""
Jul 07 12:47:17.214123 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.214101    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 12:47:17.225725 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.225704    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"]
Jul 07 12:47:17.324568 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.324531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7lzf\" (UniqueName: \"kubernetes.io/projected/3503855f-e635-4981-929b-ca0782f0b175-kube-api-access-f7lzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.324568 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.324573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.324760 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.324591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.425184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.425151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7lzf\" (UniqueName: \"kubernetes.io/projected/3503855f-e635-4981-929b-ca0782f0b175-kube-api-access-f7lzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.425341 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.425192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.425341 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.425209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.425577 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.425552    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.425638 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.425592    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.438756 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.438736    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7lzf\" (UniqueName: \"kubernetes.io/projected/3503855f-e635-4981-929b-ca0782f0b175-kube-api-access-f7lzf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.519539 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.519460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:17.646731 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.646697    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"]
Jul 07 12:47:17.649613 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:47:17.649579    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3503855f_e635_4981_929b_ca0782f0b175.slice/crio-cea76519cc760b3e1df4891c205cb9a6ca2dd3b8ff7598b592c6e6a19aa10878 WatchSource:0}: Error finding container cea76519cc760b3e1df4891c205cb9a6ca2dd3b8ff7598b592c6e6a19aa10878: Status 404 returned error can't find the container with id cea76519cc760b3e1df4891c205cb9a6ca2dd3b8ff7598b592c6e6a19aa10878
Jul 07 12:47:17.651543 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.651507    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 12:47:17.853237 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:17.853206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz" event={"ID":"3503855f-e635-4981-929b-ca0782f0b175","Type":"ContainerStarted","Data":"cea76519cc760b3e1df4891c205cb9a6ca2dd3b8ff7598b592c6e6a19aa10878"}
Jul 07 12:47:24.883752 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:24.883716    2579 generic.go:356] "Generic (PLEG): container finished" podID="3503855f-e635-4981-929b-ca0782f0b175" containerID="9e816c2cc8ab1fc72a3de6c15b4c2fe861b6a99878d1a849370e0472d53bc5bc" exitCode=0
Jul 07 12:47:24.884226 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:24.883760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz" event={"ID":"3503855f-e635-4981-929b-ca0782f0b175","Type":"ContainerDied","Data":"9e816c2cc8ab1fc72a3de6c15b4c2fe861b6a99878d1a849370e0472d53bc5bc"}
Jul 07 12:47:26.893504 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:26.893472    2579 generic.go:356] "Generic (PLEG): container finished" podID="3503855f-e635-4981-929b-ca0782f0b175" containerID="71f52205c6608a1b6f83de44eee1187922d6c3d145636b657e2c863e7da5e485" exitCode=0
Jul 07 12:47:26.893972 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:26.893568    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz" event={"ID":"3503855f-e635-4981-929b-ca0782f0b175","Type":"ContainerDied","Data":"71f52205c6608a1b6f83de44eee1187922d6c3d145636b657e2c863e7da5e485"}
Jul 07 12:47:34.927173 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:34.927135    2579 generic.go:356] "Generic (PLEG): container finished" podID="3503855f-e635-4981-929b-ca0782f0b175" containerID="caff40b4f1f18290fdf2c726b893f2a7f7b717b8ddd851d728d16d8f31505556" exitCode=0
Jul 07 12:47:34.927173 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:34.927172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz" event={"ID":"3503855f-e635-4981-929b-ca0782f0b175","Type":"ContainerDied","Data":"caff40b4f1f18290fdf2c726b893f2a7f7b717b8ddd851d728d16d8f31505556"}
Jul 07 12:47:36.058425 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.058401    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:36.178235 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.178211    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-bundle\") pod \"3503855f-e635-4981-929b-ca0782f0b175\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") "
Jul 07 12:47:36.178355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.178254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-util\") pod \"3503855f-e635-4981-929b-ca0782f0b175\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") "
Jul 07 12:47:36.178355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.178292    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7lzf\" (UniqueName: \"kubernetes.io/projected/3503855f-e635-4981-929b-ca0782f0b175-kube-api-access-f7lzf\") pod \"3503855f-e635-4981-929b-ca0782f0b175\" (UID: \"3503855f-e635-4981-929b-ca0782f0b175\") "
Jul 07 12:47:36.178872 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.178848    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-bundle" (OuterVolumeSpecName: "bundle") pod "3503855f-e635-4981-929b-ca0782f0b175" (UID: "3503855f-e635-4981-929b-ca0782f0b175"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:47:36.180431 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.180408    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3503855f-e635-4981-929b-ca0782f0b175-kube-api-access-f7lzf" (OuterVolumeSpecName: "kube-api-access-f7lzf") pod "3503855f-e635-4981-929b-ca0782f0b175" (UID: "3503855f-e635-4981-929b-ca0782f0b175"). InnerVolumeSpecName "kube-api-access-f7lzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:47:36.184379 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.184336    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-util" (OuterVolumeSpecName: "util") pod "3503855f-e635-4981-929b-ca0782f0b175" (UID: "3503855f-e635-4981-929b-ca0782f0b175"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:47:36.279252 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.279220    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:47:36.279252 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.279250    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7lzf\" (UniqueName: \"kubernetes.io/projected/3503855f-e635-4981-929b-ca0782f0b175-kube-api-access-f7lzf\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:47:36.279252 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.279261    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3503855f-e635-4981-929b-ca0782f0b175-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:47:36.936187 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.936150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz" event={"ID":"3503855f-e635-4981-929b-ca0782f0b175","Type":"ContainerDied","Data":"cea76519cc760b3e1df4891c205cb9a6ca2dd3b8ff7598b592c6e6a19aa10878"}
Jul 07 12:47:36.936349 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.936211    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cea76519cc760b3e1df4891c205cb9a6ca2dd3b8ff7598b592c6e6a19aa10878"
Jul 07 12:47:36.936349 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:36.936162    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zpnkz"
Jul 07 12:47:44.743839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.743801    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"]
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744052    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="util"
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744063    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="util"
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744071    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="extract"
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744076    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="extract"
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744094    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="pull"
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744099    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="pull"
Jul 07 12:47:44.744303 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.744141    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3503855f-e635-4981-929b-ca0782f0b175" containerName="extract"
Jul 07 12:47:44.747223 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.747207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.754997 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.754978    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjgf7\""
Jul 07 12:47:44.755122 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.755036    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 12:47:44.757956 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.757937    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 12:47:44.769895 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.769864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"]
Jul 07 12:47:44.838132 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.838087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.838315 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.838143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.838315 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.838214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vls6w\" (UniqueName: \"kubernetes.io/projected/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-kube-api-access-vls6w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.938970 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.938932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.939149 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.938977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vls6w\" (UniqueName: \"kubernetes.io/projected/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-kube-api-access-vls6w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.939149 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.939026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.939348 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.939325    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.939412 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.939394    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:44.959818 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:44.959791    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vls6w\" (UniqueName: \"kubernetes.io/projected/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-kube-api-access-vls6w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:45.056574 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:45.056450    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:45.199784 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:45.199753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"]
Jul 07 12:47:45.203254 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:47:45.203221    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4e7fb8_2883_4f7a_9993_bc579e0403a4.slice/crio-ee98051a4613ed4a7e191ee953cc75ea74211acda03e843d01de1d22f8196013 WatchSource:0}: Error finding container ee98051a4613ed4a7e191ee953cc75ea74211acda03e843d01de1d22f8196013: Status 404 returned error can't find the container with id ee98051a4613ed4a7e191ee953cc75ea74211acda03e843d01de1d22f8196013
Jul 07 12:47:45.972038 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:45.972001    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerID="e8bdd3a6a90943f48222932a62d50fecfd1c0a6c3ab94176f7a311730259ed86" exitCode=0
Jul 07 12:47:45.972416 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:45.972085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq" event={"ID":"bf4e7fb8-2883-4f7a-9993-bc579e0403a4","Type":"ContainerDied","Data":"e8bdd3a6a90943f48222932a62d50fecfd1c0a6c3ab94176f7a311730259ed86"}
Jul 07 12:47:45.972416 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:45.972139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq" event={"ID":"bf4e7fb8-2883-4f7a-9993-bc579e0403a4","Type":"ContainerStarted","Data":"ee98051a4613ed4a7e191ee953cc75ea74211acda03e843d01de1d22f8196013"}
Jul 07 12:47:48.984592 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:48.984554    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerID="76cfbf67910beadf79d74b35e41a0a108f84654852eb6f31bc54cf29fd91a4fa" exitCode=0
Jul 07 12:47:48.984979 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:48.984644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq" event={"ID":"bf4e7fb8-2883-4f7a-9993-bc579e0403a4","Type":"ContainerDied","Data":"76cfbf67910beadf79d74b35e41a0a108f84654852eb6f31bc54cf29fd91a4fa"}
Jul 07 12:47:49.990219 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:49.990185    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerID="739f397f1cc9ba84b4ee8579cf23ae8e3263c8fd9e11b460911f1e5ebb976d44" exitCode=0
Jul 07 12:47:49.990634 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:49.990234    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq" event={"ID":"bf4e7fb8-2883-4f7a-9993-bc579e0403a4","Type":"ContainerDied","Data":"739f397f1cc9ba84b4ee8579cf23ae8e3263c8fd9e11b460911f1e5ebb976d44"}
Jul 07 12:47:51.116711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.116686    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:51.186351 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.186316    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vls6w\" (UniqueName: \"kubernetes.io/projected/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-kube-api-access-vls6w\") pod \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") "
Jul 07 12:47:51.186495 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.186401    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-bundle\") pod \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") "
Jul 07 12:47:51.186495 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.186430    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-util\") pod \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\" (UID: \"bf4e7fb8-2883-4f7a-9993-bc579e0403a4\") "
Jul 07 12:47:51.186840 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.186806    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-bundle" (OuterVolumeSpecName: "bundle") pod "bf4e7fb8-2883-4f7a-9993-bc579e0403a4" (UID: "bf4e7fb8-2883-4f7a-9993-bc579e0403a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:47:51.188488 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.188459    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-kube-api-access-vls6w" (OuterVolumeSpecName: "kube-api-access-vls6w") pod "bf4e7fb8-2883-4f7a-9993-bc579e0403a4" (UID: "bf4e7fb8-2883-4f7a-9993-bc579e0403a4"). InnerVolumeSpecName "kube-api-access-vls6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:47:51.191632 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.191598    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-util" (OuterVolumeSpecName: "util") pod "bf4e7fb8-2883-4f7a-9993-bc579e0403a4" (UID: "bf4e7fb8-2883-4f7a-9993-bc579e0403a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:47:51.286973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.286884    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:47:51.286973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.286916    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:47:51.286973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.286925    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vls6w\" (UniqueName: \"kubernetes.io/projected/bf4e7fb8-2883-4f7a-9993-bc579e0403a4-kube-api-access-vls6w\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:47:51.999311 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.999272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq" event={"ID":"bf4e7fb8-2883-4f7a-9993-bc579e0403a4","Type":"ContainerDied","Data":"ee98051a4613ed4a7e191ee953cc75ea74211acda03e843d01de1d22f8196013"}
Jul 07 12:47:51.999311 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.999312    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrjgq"
Jul 07 12:47:51.999541 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:51.999349    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee98051a4613ed4a7e191ee953cc75ea74211acda03e843d01de1d22f8196013"
Jul 07 12:47:57.839224 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839191    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"]
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839467    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="extract"
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839478    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="extract"
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839491    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="util"
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839497    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="util"
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839504    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="pull"
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839509    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="pull"
Jul 07 12:47:57.839657 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.839587    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4e7fb8-2883-4f7a-9993-bc579e0403a4" containerName="extract"
Jul 07 12:47:57.841381 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.841364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:57.843592 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.843572    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 12:47:57.844058 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.844040    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zkssm\""
Jul 07 12:47:57.844165 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.844066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 12:47:57.855841 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.852638    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"]
Jul 07 12:47:57.937982 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.937940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515510f9-cd07-4fa3-b3da-493612290a4a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gbblb\" (UID: \"515510f9-cd07-4fa3-b3da-493612290a4a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:57.938144 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:57.937991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz22v\" (UniqueName: \"kubernetes.io/projected/515510f9-cd07-4fa3-b3da-493612290a4a-kube-api-access-kz22v\") pod \"openshift-lws-operator-fd8ccff4c-gbblb\" (UID: \"515510f9-cd07-4fa3-b3da-493612290a4a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:58.039327 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:58.039286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515510f9-cd07-4fa3-b3da-493612290a4a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gbblb\" (UID: \"515510f9-cd07-4fa3-b3da-493612290a4a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:58.039327 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:58.039331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz22v\" (UniqueName: \"kubernetes.io/projected/515510f9-cd07-4fa3-b3da-493612290a4a-kube-api-access-kz22v\") pod \"openshift-lws-operator-fd8ccff4c-gbblb\" (UID: \"515510f9-cd07-4fa3-b3da-493612290a4a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:58.039735 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:58.039714    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/515510f9-cd07-4fa3-b3da-493612290a4a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gbblb\" (UID: \"515510f9-cd07-4fa3-b3da-493612290a4a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:58.048762 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:58.048731    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz22v\" (UniqueName: \"kubernetes.io/projected/515510f9-cd07-4fa3-b3da-493612290a4a-kube-api-access-kz22v\") pod \"openshift-lws-operator-fd8ccff4c-gbblb\" (UID: \"515510f9-cd07-4fa3-b3da-493612290a4a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:58.150655 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:58.150571    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"
Jul 07 12:47:58.283973 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:58.283934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb"]
Jul 07 12:47:58.287931 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:47:58.287892    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod515510f9_cd07_4fa3_b3da_493612290a4a.slice/crio-2786b9c157e8a9d841a36d48a98246bac31dac94ce9e7df8008dabb5fd2d5aec WatchSource:0}: Error finding container 2786b9c157e8a9d841a36d48a98246bac31dac94ce9e7df8008dabb5fd2d5aec: Status 404 returned error can't find the container with id 2786b9c157e8a9d841a36d48a98246bac31dac94ce9e7df8008dabb5fd2d5aec
Jul 07 12:47:59.029113 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:47:59.029082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb" event={"ID":"515510f9-cd07-4fa3-b3da-493612290a4a","Type":"ContainerStarted","Data":"2786b9c157e8a9d841a36d48a98246bac31dac94ce9e7df8008dabb5fd2d5aec"}
Jul 07 12:48:01.039191 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:01.039152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb" event={"ID":"515510f9-cd07-4fa3-b3da-493612290a4a","Type":"ContainerStarted","Data":"1e2e4ae260c0134866a24eda4e0d5db59c18dba6cc5d253505706d89b8d7d290"}
Jul 07 12:48:01.056129 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:01.056057    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gbblb" podStartSLOduration=1.950734484 podStartE2EDuration="4.056033283s" totalImagesPullingTime="2.105298799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:01.055178834 +0000 UTC m=+349.056045753" watchObservedRunningTime="2026-07-07 12:48:01.056033283 +0000 UTC m=+349.056900201"
Jul 07 12:48:04.129047 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.128999    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"]
Jul 07 12:48:04.132727 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.132699    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.136467 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.136443    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 12:48:04.136646 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.136503    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjgf7\""
Jul 07 12:48:04.136964 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.136944    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 12:48:04.148532 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.148482    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"]
Jul 07 12:48:04.186512 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.186477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.186703 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.186545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.186703 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.186672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4kk7\" (UniqueName: \"kubernetes.io/projected/3553692c-3f6b-47d1-a56e-5f871ad513ac-kube-api-access-z4kk7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.287931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.287842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4kk7\" (UniqueName: \"kubernetes.io/projected/3553692c-3f6b-47d1-a56e-5f871ad513ac-kube-api-access-z4kk7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.287931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.287897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.288152 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.287937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.288323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.288303    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.288379 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.288325    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.301044 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.301000    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4kk7\" (UniqueName: \"kubernetes.io/projected/3553692c-3f6b-47d1-a56e-5f871ad513ac-kube-api-access-z4kk7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.441834 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.441744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:04.586427 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:04.586391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"]
Jul 07 12:48:04.589807 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:48:04.589771    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3553692c_3f6b_47d1_a56e_5f871ad513ac.slice/crio-89bb03182f79db420d9f4403b19b20394f26d4bc6684ed0a1cc5d5ed57bbb16f WatchSource:0}: Error finding container 89bb03182f79db420d9f4403b19b20394f26d4bc6684ed0a1cc5d5ed57bbb16f: Status 404 returned error can't find the container with id 89bb03182f79db420d9f4403b19b20394f26d4bc6684ed0a1cc5d5ed57bbb16f
Jul 07 12:48:05.055995 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:05.055907    2579 generic.go:356] "Generic (PLEG): container finished" podID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerID="57b79b36548b148b1c36debd0f997c83aba88c34770952d92c3537519db72320" exitCode=0
Jul 07 12:48:05.055995 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:05.055976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4" event={"ID":"3553692c-3f6b-47d1-a56e-5f871ad513ac","Type":"ContainerDied","Data":"57b79b36548b148b1c36debd0f997c83aba88c34770952d92c3537519db72320"}
Jul 07 12:48:05.056190 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:05.056021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4" event={"ID":"3553692c-3f6b-47d1-a56e-5f871ad513ac","Type":"ContainerStarted","Data":"89bb03182f79db420d9f4403b19b20394f26d4bc6684ed0a1cc5d5ed57bbb16f"}
Jul 07 12:48:06.061712 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:06.061684    2579 generic.go:356] "Generic (PLEG): container finished" podID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerID="99347928acb8db8071fa63b453d465161a90c95255fbb28f88b1db9491f8d7eb" exitCode=0
Jul 07 12:48:06.062093 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:06.061752    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4" event={"ID":"3553692c-3f6b-47d1-a56e-5f871ad513ac","Type":"ContainerDied","Data":"99347928acb8db8071fa63b453d465161a90c95255fbb28f88b1db9491f8d7eb"}
Jul 07 12:48:07.066806 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:07.066773    2579 generic.go:356] "Generic (PLEG): container finished" podID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerID="14b671ce8274502aaecfc8a520abe7c75198be78acaee6f4a331152c53e489d7" exitCode=0
Jul 07 12:48:07.067186 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:07.066815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4" event={"ID":"3553692c-3f6b-47d1-a56e-5f871ad513ac","Type":"ContainerDied","Data":"14b671ce8274502aaecfc8a520abe7c75198be78acaee6f4a331152c53e489d7"}
Jul 07 12:48:08.201529 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.201488    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:08.322820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.322720    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-util\") pod \"3553692c-3f6b-47d1-a56e-5f871ad513ac\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") "
Jul 07 12:48:08.322820 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.322772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4kk7\" (UniqueName: \"kubernetes.io/projected/3553692c-3f6b-47d1-a56e-5f871ad513ac-kube-api-access-z4kk7\") pod \"3553692c-3f6b-47d1-a56e-5f871ad513ac\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") "
Jul 07 12:48:08.323057 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.322833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-bundle\") pod \"3553692c-3f6b-47d1-a56e-5f871ad513ac\" (UID: \"3553692c-3f6b-47d1-a56e-5f871ad513ac\") "
Jul 07 12:48:08.323575 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.323552    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-bundle" (OuterVolumeSpecName: "bundle") pod "3553692c-3f6b-47d1-a56e-5f871ad513ac" (UID: "3553692c-3f6b-47d1-a56e-5f871ad513ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:48:08.324941 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.324910    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3553692c-3f6b-47d1-a56e-5f871ad513ac-kube-api-access-z4kk7" (OuterVolumeSpecName: "kube-api-access-z4kk7") pod "3553692c-3f6b-47d1-a56e-5f871ad513ac" (UID: "3553692c-3f6b-47d1-a56e-5f871ad513ac"). InnerVolumeSpecName "kube-api-access-z4kk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:48:08.328091 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.328068    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-util" (OuterVolumeSpecName: "util") pod "3553692c-3f6b-47d1-a56e-5f871ad513ac" (UID: "3553692c-3f6b-47d1-a56e-5f871ad513ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:48:08.423642 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.423594    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:48:08.423642 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.423633    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3553692c-3f6b-47d1-a56e-5f871ad513ac-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:48:08.423642 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:08.423644    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4kk7\" (UniqueName: \"kubernetes.io/projected/3553692c-3f6b-47d1-a56e-5f871ad513ac-kube-api-access-z4kk7\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:48:09.076441 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:09.076411    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4"
Jul 07 12:48:09.076441 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:09.076422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afknnw4" event={"ID":"3553692c-3f6b-47d1-a56e-5f871ad513ac","Type":"ContainerDied","Data":"89bb03182f79db420d9f4403b19b20394f26d4bc6684ed0a1cc5d5ed57bbb16f"}
Jul 07 12:48:09.076672 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:09.076465    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89bb03182f79db420d9f4403b19b20394f26d4bc6684ed0a1cc5d5ed57bbb16f"
Jul 07 12:48:16.971695 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971402    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"]
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971782    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="util"
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971800    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="util"
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971809    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="extract"
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971817    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="extract"
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971843    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="pull"
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971850    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="pull"
Jul 07 12:48:16.972156 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.971906    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3553692c-3f6b-47d1-a56e-5f871ad513ac" containerName="extract"
Jul 07 12:48:16.976390 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.976364    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"]
Jul 07 12:48:16.976563 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.976510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:16.987825 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.987799    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7tjqs\""
Jul 07 12:48:16.989136 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.989107    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 12:48:16.990857 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.990839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 12:48:16.991563 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.991548    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 12:48:16.991849 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:16.991836    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 12:48:17.087346 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.087302    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4aac462d-57f4-445e-ab64-daf261410896-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.087566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.087381    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4aac462d-57f4-445e-ab64-daf261410896-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.087566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.087409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4aac462d-57f4-445e-ab64-daf261410896-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.087566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.087448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwwr\" (UniqueName: \"kubernetes.io/projected/4aac462d-57f4-445e-ab64-daf261410896-kube-api-access-xbwwr\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.087566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.087499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4aac462d-57f4-445e-ab64-daf261410896-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.188134 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.188089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4aac462d-57f4-445e-ab64-daf261410896-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.188355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.188138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4aac462d-57f4-445e-ab64-daf261410896-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.188355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.188190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwwr\" (UniqueName: \"kubernetes.io/projected/4aac462d-57f4-445e-ab64-daf261410896-kube-api-access-xbwwr\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.188355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.188225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4aac462d-57f4-445e-ab64-daf261410896-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.188355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.188251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4aac462d-57f4-445e-ab64-daf261410896-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.188693 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.188668    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4aac462d-57f4-445e-ab64-daf261410896-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.190697 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.190666    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4aac462d-57f4-445e-ab64-daf261410896-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.190918 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.190898    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4aac462d-57f4-445e-ab64-daf261410896-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.190961 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.190897    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4aac462d-57f4-445e-ab64-daf261410896-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.204565 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.204510    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwwr\" (UniqueName: \"kubernetes.io/projected/4aac462d-57f4-445e-ab64-daf261410896-kube-api-access-xbwwr\") pod \"opendatahub-operator-controller-manager-68cfd86456-d84kw\" (UID: \"4aac462d-57f4-445e-ab64-daf261410896\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.286488 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.286393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:17.485494 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:17.485458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"]
Jul 07 12:48:17.501180 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:48:17.501147    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aac462d_57f4_445e_ab64_daf261410896.slice/crio-833e8ea08faf6c0060aae36a312a5a7d9dd5c87cbbe924a20b50a78b469bbfc6 WatchSource:0}: Error finding container 833e8ea08faf6c0060aae36a312a5a7d9dd5c87cbbe924a20b50a78b469bbfc6: Status 404 returned error can't find the container with id 833e8ea08faf6c0060aae36a312a5a7d9dd5c87cbbe924a20b50a78b469bbfc6
Jul 07 12:48:18.115271 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:18.115234    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw" event={"ID":"4aac462d-57f4-445e-ab64-daf261410896","Type":"ContainerStarted","Data":"833e8ea08faf6c0060aae36a312a5a7d9dd5c87cbbe924a20b50a78b469bbfc6"}
Jul 07 12:48:21.131366 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:21.131327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw" event={"ID":"4aac462d-57f4-445e-ab64-daf261410896","Type":"ContainerStarted","Data":"bcaa0267b93c6b81124a52931d4b46358af7e4f0f03bcb788bb2b2c5c42fdd1b"}
Jul 07 12:48:22.138372 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:22.138338    2579 generic.go:356] "Generic (PLEG): container finished" podID="4aac462d-57f4-445e-ab64-daf261410896" containerID="bcaa0267b93c6b81124a52931d4b46358af7e4f0f03bcb788bb2b2c5c42fdd1b" exitCode=0
Jul 07 12:48:22.138832 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:22.138412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw" event={"ID":"4aac462d-57f4-445e-ab64-daf261410896","Type":"ContainerDied","Data":"bcaa0267b93c6b81124a52931d4b46358af7e4f0f03bcb788bb2b2c5c42fdd1b"}
Jul 07 12:48:23.143734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:23.143700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw" event={"ID":"4aac462d-57f4-445e-ab64-daf261410896","Type":"ContainerStarted","Data":"733d03aafa878cb9fbb83b6f6e19d2f02e2d0bacc9dd6357437badf0226c5eaa"}
Jul 07 12:48:23.144135 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:23.143806    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:23.165604 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:23.165553    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw" podStartSLOduration=2.689201094 podStartE2EDuration="7.16553108s" totalImagesPullingTime="3.3830321s" totalInitContainerRuntime="1.093297886s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:23.163768007 +0000 UTC m=+371.164634928" watchObservedRunningTime="2026-07-07 12:48:23.16553108 +0000 UTC m=+371.166397992"
Jul 07 12:48:34.150434 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.150397    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-d84kw"
Jul 07 12:48:34.183388 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.183343    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"]
Jul 07 12:48:34.193035 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.193010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.198118 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.198088    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 12:48:34.198754 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.198725    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hhbbh\""
Jul 07 12:48:34.199481 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.199211    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 12:48:34.199580 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.199562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 12:48:34.217348 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.217313    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"]
Jul 07 12:48:34.329459 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.329419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-metrics-cert\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.329667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.329483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjqmf\" (UniqueName: \"kubernetes.io/projected/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-kube-api-access-tjqmf\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.329667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.329543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-cert\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.329667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.329563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-manager-config\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.430497 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.430407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-cert\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.430497 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.430446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-manager-config\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.430497 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.430476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-metrics-cert\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.430742 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.430553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjqmf\" (UniqueName: \"kubernetes.io/projected/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-kube-api-access-tjqmf\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.431112 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.431079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-manager-config\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.433036 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.433011    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-cert\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.433135 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.433055    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-metrics-cert\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.441923 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.441891    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjqmf\" (UniqueName: \"kubernetes.io/projected/81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447-kube-api-access-tjqmf\") pod \"lws-controller-manager-65fd447f77-pmgdw\" (UID: \"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.506752 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.506714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:34.673783 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:34.673749    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"]
Jul 07 12:48:34.676759 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:48:34.676728    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f6f2ce_2d2a_401b_9b25_d2ca9f3d4447.slice/crio-f0f74d70a76b78be4546cce781f26bf051955739089a5486b97905420946b278 WatchSource:0}: Error finding container f0f74d70a76b78be4546cce781f26bf051955739089a5486b97905420946b278: Status 404 returned error can't find the container with id f0f74d70a76b78be4546cce781f26bf051955739089a5486b97905420946b278
Jul 07 12:48:35.195947 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:35.195903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw" event={"ID":"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447","Type":"ContainerStarted","Data":"f0f74d70a76b78be4546cce781f26bf051955739089a5486b97905420946b278"}
Jul 07 12:48:37.611408 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.611372    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"]
Jul 07 12:48:37.614162 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.614141    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.620164 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.620137    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 12:48:37.620331 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.620164    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 12:48:37.621357 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.620964    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 12:48:37.622002 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.621981    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n2k64\""
Jul 07 12:48:37.623745 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.623725    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 12:48:37.643605 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.643572    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"]
Jul 07 12:48:37.760101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.760062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69959d63-f09d-448c-811c-9c948a867a39-tmp\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.760294 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.760130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69959d63-f09d-448c-811c-9c948a867a39-tls-certs\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.760294 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.760171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6tnz\" (UniqueName: \"kubernetes.io/projected/69959d63-f09d-448c-811c-9c948a867a39-kube-api-access-f6tnz\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.860858 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.860815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69959d63-f09d-448c-811c-9c948a867a39-tls-certs\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.861029 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.860870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6tnz\" (UniqueName: \"kubernetes.io/projected/69959d63-f09d-448c-811c-9c948a867a39-kube-api-access-f6tnz\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.861029 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.860897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69959d63-f09d-448c-811c-9c948a867a39-tmp\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.863213 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.863151    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69959d63-f09d-448c-811c-9c948a867a39-tmp\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.863375 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.863355    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69959d63-f09d-448c-811c-9c948a867a39-tls-certs\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.898860 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.898821    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6tnz\" (UniqueName: \"kubernetes.io/projected/69959d63-f09d-448c-811c-9c948a867a39-kube-api-access-f6tnz\") pod \"kube-auth-proxy-74684786d9-4ld4x\" (UID: \"69959d63-f09d-448c-811c-9c948a867a39\") " pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:37.924633 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:37.924590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"
Jul 07 12:48:38.097132 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:38.097098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74684786d9-4ld4x"]
Jul 07 12:48:38.100669 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:48:38.100628    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69959d63_f09d_448c_811c_9c948a867a39.slice/crio-b93fa57ce57b03175cfbabc17af7b928265381673371fa4f05ea96c063b9fdf2 WatchSource:0}: Error finding container b93fa57ce57b03175cfbabc17af7b928265381673371fa4f05ea96c063b9fdf2: Status 404 returned error can't find the container with id b93fa57ce57b03175cfbabc17af7b928265381673371fa4f05ea96c063b9fdf2
Jul 07 12:48:38.211502 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:38.211410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x" event={"ID":"69959d63-f09d-448c-811c-9c948a867a39","Type":"ContainerStarted","Data":"b93fa57ce57b03175cfbabc17af7b928265381673371fa4f05ea96c063b9fdf2"}
Jul 07 12:48:41.225211 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:41.225123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x" event={"ID":"69959d63-f09d-448c-811c-9c948a867a39","Type":"ContainerStarted","Data":"521412ec5efffc763a5d81c0be94f57bcb90eeb53cede8d8fa5ad6d83b865937"}
Jul 07 12:48:41.268273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:41.268229    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74684786d9-4ld4x" podStartSLOduration=1.5037798489999998 podStartE2EDuration="4.268213394s" totalImagesPullingTime="2.764433545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:41.26571874 +0000 UTC m=+389.266585658" watchObservedRunningTime="2026-07-07 12:48:41.268213394 +0000 UTC m=+389.269080314"
Jul 07 12:48:43.236464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:43.236423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw" event={"ID":"81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447","Type":"ContainerStarted","Data":"592b5a25076f6f555ff4fa20b36776639b5f72e8c646b7bf5da2e0a995cb696f"}
Jul 07 12:48:43.236873 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:43.236482    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:48:43.273865 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:43.273812    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw" podStartSLOduration=1.295009146 podStartE2EDuration="9.273796323s" totalImagesPullingTime="7.978787177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:43.272298553 +0000 UTC m=+391.273165472" watchObservedRunningTime="2026-07-07 12:48:43.273796323 +0000 UTC m=+391.274663241"
Jul 07 12:48:54.244449 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:48:54.244421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-pmgdw"
Jul 07 12:49:54.108495 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.108458    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"]
Jul 07 12:49:54.110805 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.110784    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.115078 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.115048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qlvmx\""
Jul 07 12:49:54.115209 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.115192    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 12:49:54.115294 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.115279    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 12:49:54.127305 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.127281    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"]
Jul 07 12:49:54.216877 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.216840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.216877 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.216882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db6ww\" (UniqueName: \"kubernetes.io/projected/ce07c240-0c91-455e-9fed-3174530ab721-kube-api-access-db6ww\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.217129 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.216928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.317681 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.317645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db6ww\" (UniqueName: \"kubernetes.io/projected/ce07c240-0c91-455e-9fed-3174530ab721-kube-api-access-db6ww\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.317860 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.317733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.317860 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.317775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.318114 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.318095    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.318183 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.318131    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.329943 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.329921    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db6ww\" (UniqueName: \"kubernetes.io/projected/ce07c240-0c91-455e-9fed-3174530ab721-kube-api-access-db6ww\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.420065 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.419999    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:54.553247 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.553222    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"]
Jul 07 12:49:54.555832 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:49:54.555806    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce07c240_0c91_455e_9fed_3174530ab721.slice/crio-e763f17f05c595596c1daaea13de4ca84955632868aad383648b94ff8a831ee7 WatchSource:0}: Error finding container e763f17f05c595596c1daaea13de4ca84955632868aad383648b94ff8a831ee7: Status 404 returned error can't find the container with id e763f17f05c595596c1daaea13de4ca84955632868aad383648b94ff8a831ee7
Jul 07 12:49:54.704314 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.704240    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"]
Jul 07 12:49:54.706630 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.706612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.757903 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.757875    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"]
Jul 07 12:49:54.821245 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.821215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.821397 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.821256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.821397 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.821279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qt6m\" (UniqueName: \"kubernetes.io/projected/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-kube-api-access-7qt6m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.921888 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.921852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.921888 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.921892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qt6m\" (UniqueName: \"kubernetes.io/projected/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-kube-api-access-7qt6m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.922108 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.921948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.922341 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.922318    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.922341 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.922331    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:54.931151 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:54.931128    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qt6m\" (UniqueName: \"kubernetes.io/projected/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-kube-api-access-7qt6m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:55.024260 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.024170    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:55.174270 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.174009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"]
Jul 07 12:49:55.178702 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:49:55.178669    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45cc3af_0bb3_4eaa_b798_75e4ff60c012.slice/crio-18e570dd407e0b15a0bea25c1e5150b7c173038533849a214272fa6456fd204e WatchSource:0}: Error finding container 18e570dd407e0b15a0bea25c1e5150b7c173038533849a214272fa6456fd204e: Status 404 returned error can't find the container with id 18e570dd407e0b15a0bea25c1e5150b7c173038533849a214272fa6456fd204e
Jul 07 12:49:55.225938 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.225908    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"]
Jul 07 12:49:55.228996 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.228979    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.256680 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.256643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"]
Jul 07 12:49:55.325560 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.325501    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45r2t\" (UniqueName: \"kubernetes.io/projected/26b6855d-2915-4943-a742-35629e91fe52-kube-api-access-45r2t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.325758 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.325587    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.325758 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.325627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.427105 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.427066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.427310 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.427115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.427310 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.427167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45r2t\" (UniqueName: \"kubernetes.io/projected/26b6855d-2915-4943-a742-35629e91fe52-kube-api-access-45r2t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.427626 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.427602    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.427701 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.427606    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.439988 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.439957    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45r2t\" (UniqueName: \"kubernetes.io/projected/26b6855d-2915-4943-a742-35629e91fe52-kube-api-access-45r2t\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.540066 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.539975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:49:55.543482 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.543458    2579 generic.go:356] "Generic (PLEG): container finished" podID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerID="6e4b3ba6a8845cb75151660e2d84df896fc2456b5e82611d3b79486e018b7c3e" exitCode=0
Jul 07 12:49:55.543563 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.543537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp" event={"ID":"e45cc3af-0bb3-4eaa-b798-75e4ff60c012","Type":"ContainerDied","Data":"6e4b3ba6a8845cb75151660e2d84df896fc2456b5e82611d3b79486e018b7c3e"}
Jul 07 12:49:55.543606 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.543593    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp" event={"ID":"e45cc3af-0bb3-4eaa-b798-75e4ff60c012","Type":"ContainerStarted","Data":"18e570dd407e0b15a0bea25c1e5150b7c173038533849a214272fa6456fd204e"}
Jul 07 12:49:55.545184 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.545161    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce07c240-0c91-455e-9fed-3174530ab721" containerID="e6126b50245d72b15c954ec4abb315d80249d0018c8aab93a1ffbb2fb0ec4778" exitCode=0
Jul 07 12:49:55.545281 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.545232    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd" event={"ID":"ce07c240-0c91-455e-9fed-3174530ab721","Type":"ContainerDied","Data":"e6126b50245d72b15c954ec4abb315d80249d0018c8aab93a1ffbb2fb0ec4778"}
Jul 07 12:49:55.545281 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.545276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd" event={"ID":"ce07c240-0c91-455e-9fed-3174530ab721","Type":"ContainerStarted","Data":"e763f17f05c595596c1daaea13de4ca84955632868aad383648b94ff8a831ee7"}
Jul 07 12:49:55.574480 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.574448    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"]
Jul 07 12:49:55.577847 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.577824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.612980 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.612944    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"]
Jul 07 12:49:55.628848 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.628824    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nh25\" (UniqueName: \"kubernetes.io/projected/3d16359c-14b0-49af-a484-16d703780bd2-kube-api-access-9nh25\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.628962 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.628890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.628962 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.628920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.689032 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.689000    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"]
Jul 07 12:49:55.691872 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:49:55.691830    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b6855d_2915_4943_a742_35629e91fe52.slice/crio-c116530510a6769e39f8c676c360d157d3752b5cfe1880bf7abf73b897d2fbba WatchSource:0}: Error finding container c116530510a6769e39f8c676c360d157d3752b5cfe1880bf7abf73b897d2fbba: Status 404 returned error can't find the container with id c116530510a6769e39f8c676c360d157d3752b5cfe1880bf7abf73b897d2fbba
Jul 07 12:49:55.729660 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.729621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nh25\" (UniqueName: \"kubernetes.io/projected/3d16359c-14b0-49af-a484-16d703780bd2-kube-api-access-9nh25\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.729817 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.729670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.729817 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.729710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.730095 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.730077    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.730145 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.730123    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.740692 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.740672    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nh25\" (UniqueName: \"kubernetes.io/projected/3d16359c-14b0-49af-a484-16d703780bd2-kube-api-access-9nh25\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:55.889124 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:55.889081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:56.034677 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.034648    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"]
Jul 07 12:49:56.037665 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:49:56.037631    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d16359c_14b0_49af_a484_16d703780bd2.slice/crio-08b981ec021a18175fbb4bd48d095f01cef5215ebd59cadad5a873d9216bbe93 WatchSource:0}: Error finding container 08b981ec021a18175fbb4bd48d095f01cef5215ebd59cadad5a873d9216bbe93: Status 404 returned error can't find the container with id 08b981ec021a18175fbb4bd48d095f01cef5215ebd59cadad5a873d9216bbe93
Jul 07 12:49:56.551945 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.551910    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d16359c-14b0-49af-a484-16d703780bd2" containerID="552eb0fb1086252773baa7688934fa7225b016a0348f5bce2bb349e33fa1dd2d" exitCode=0
Jul 07 12:49:56.552370 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.551998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x" event={"ID":"3d16359c-14b0-49af-a484-16d703780bd2","Type":"ContainerDied","Data":"552eb0fb1086252773baa7688934fa7225b016a0348f5bce2bb349e33fa1dd2d"}
Jul 07 12:49:56.552370 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.552065    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x" event={"ID":"3d16359c-14b0-49af-a484-16d703780bd2","Type":"ContainerStarted","Data":"08b981ec021a18175fbb4bd48d095f01cef5215ebd59cadad5a873d9216bbe93"}
Jul 07 12:49:56.553610 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.553582    2579 generic.go:356] "Generic (PLEG): container finished" podID="26b6855d-2915-4943-a742-35629e91fe52" containerID="9d74e0e1bc8aa169927a27faa38b4550165d06b2b5e87665c368d35ebf45dd6c" exitCode=0
Jul 07 12:49:56.553746 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.553674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh" event={"ID":"26b6855d-2915-4943-a742-35629e91fe52","Type":"ContainerDied","Data":"9d74e0e1bc8aa169927a27faa38b4550165d06b2b5e87665c368d35ebf45dd6c"}
Jul 07 12:49:56.553746 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.553734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh" event={"ID":"26b6855d-2915-4943-a742-35629e91fe52","Type":"ContainerStarted","Data":"c116530510a6769e39f8c676c360d157d3752b5cfe1880bf7abf73b897d2fbba"}
Jul 07 12:49:56.555765 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.555741    2579 generic.go:356] "Generic (PLEG): container finished" podID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerID="dd29925899ce2fe45efb50721ff0e7803fb11aa56e0fc46447c84e96c17fda7f" exitCode=0
Jul 07 12:49:56.555842 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:56.555818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp" event={"ID":"e45cc3af-0bb3-4eaa-b798-75e4ff60c012","Type":"ContainerDied","Data":"dd29925899ce2fe45efb50721ff0e7803fb11aa56e0fc46447c84e96c17fda7f"}
Jul 07 12:49:57.563080 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.563047    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce07c240-0c91-455e-9fed-3174530ab721" containerID="4aa57b4b5fb0b30729fc09e7964f8cc96700ba1ca13fa8553b7eafdf405418cf" exitCode=0
Jul 07 12:49:57.563656 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.563124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd" event={"ID":"ce07c240-0c91-455e-9fed-3174530ab721","Type":"ContainerDied","Data":"4aa57b4b5fb0b30729fc09e7964f8cc96700ba1ca13fa8553b7eafdf405418cf"}
Jul 07 12:49:57.565133 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.565108    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d16359c-14b0-49af-a484-16d703780bd2" containerID="bdf5f8e2ababcd9259b1d6542515cb7669ff2f69a5f5f001a666edfcd15b08c7" exitCode=0
Jul 07 12:49:57.565254 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.565188    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x" event={"ID":"3d16359c-14b0-49af-a484-16d703780bd2","Type":"ContainerDied","Data":"bdf5f8e2ababcd9259b1d6542515cb7669ff2f69a5f5f001a666edfcd15b08c7"}
Jul 07 12:49:57.567200 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.567175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh" event={"ID":"26b6855d-2915-4943-a742-35629e91fe52","Type":"ContainerStarted","Data":"b16c6c3224997963f3d8cea7a3a3af9d66ba40cdc8f18c056333e546d7c8acba"}
Jul 07 12:49:57.569658 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.569510    2579 generic.go:356] "Generic (PLEG): container finished" podID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerID="940b6b81ca643754abda55bc1a048607f09d28c1307ae17817e8f1f9d82e0621" exitCode=0
Jul 07 12:49:57.569658 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:57.569619    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp" event={"ID":"e45cc3af-0bb3-4eaa-b798-75e4ff60c012","Type":"ContainerDied","Data":"940b6b81ca643754abda55bc1a048607f09d28c1307ae17817e8f1f9d82e0621"}
Jul 07 12:49:58.577216 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.577173    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce07c240-0c91-455e-9fed-3174530ab721" containerID="ac490561b24e4d3d8de3fdd7f5a4a2377165577ff8b9509400f09ab123cd63db" exitCode=0
Jul 07 12:49:58.577696 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.577249    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd" event={"ID":"ce07c240-0c91-455e-9fed-3174530ab721","Type":"ContainerDied","Data":"ac490561b24e4d3d8de3fdd7f5a4a2377165577ff8b9509400f09ab123cd63db"}
Jul 07 12:49:58.579131 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.579108    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d16359c-14b0-49af-a484-16d703780bd2" containerID="c7263595e16c79f690f6da114ac820def7edffb3f0a476cb4302eef6d0e13d81" exitCode=0
Jul 07 12:49:58.579229 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.579171    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x" event={"ID":"3d16359c-14b0-49af-a484-16d703780bd2","Type":"ContainerDied","Data":"c7263595e16c79f690f6da114ac820def7edffb3f0a476cb4302eef6d0e13d81"}
Jul 07 12:49:58.580791 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.580772    2579 generic.go:356] "Generic (PLEG): container finished" podID="26b6855d-2915-4943-a742-35629e91fe52" containerID="b16c6c3224997963f3d8cea7a3a3af9d66ba40cdc8f18c056333e546d7c8acba" exitCode=0
Jul 07 12:49:58.580898 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.580851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh" event={"ID":"26b6855d-2915-4943-a742-35629e91fe52","Type":"ContainerDied","Data":"b16c6c3224997963f3d8cea7a3a3af9d66ba40cdc8f18c056333e546d7c8acba"}
Jul 07 12:49:58.721883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.721856    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:58.857666 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.857563    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-bundle\") pod \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") "
Jul 07 12:49:58.857666 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.857671    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-util\") pod \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") "
Jul 07 12:49:58.857905 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.857707    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qt6m\" (UniqueName: \"kubernetes.io/projected/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-kube-api-access-7qt6m\") pod \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\" (UID: \"e45cc3af-0bb3-4eaa-b798-75e4ff60c012\") "
Jul 07 12:49:58.858214 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.858187    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-bundle" (OuterVolumeSpecName: "bundle") pod "e45cc3af-0bb3-4eaa-b798-75e4ff60c012" (UID: "e45cc3af-0bb3-4eaa-b798-75e4ff60c012"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:49:58.859963 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.859936    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-kube-api-access-7qt6m" (OuterVolumeSpecName: "kube-api-access-7qt6m") pod "e45cc3af-0bb3-4eaa-b798-75e4ff60c012" (UID: "e45cc3af-0bb3-4eaa-b798-75e4ff60c012"). InnerVolumeSpecName "kube-api-access-7qt6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:49:58.863229 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.863209    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-util" (OuterVolumeSpecName: "util") pod "e45cc3af-0bb3-4eaa-b798-75e4ff60c012" (UID: "e45cc3af-0bb3-4eaa-b798-75e4ff60c012"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:49:58.958946 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.958917    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:58.958946 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.958944    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qt6m\" (UniqueName: \"kubernetes.io/projected/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-kube-api-access-7qt6m\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:58.959101 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:58.958957    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e45cc3af-0bb3-4eaa-b798-75e4ff60c012-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:59.587128 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.587038    2579 generic.go:356] "Generic (PLEG): container finished" podID="26b6855d-2915-4943-a742-35629e91fe52" containerID="3ebab82011384c1a2f79a9117651ac9daa581a8255e1edcea5a949e40cbc86d5" exitCode=0
Jul 07 12:49:59.587555 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.587122    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh" event={"ID":"26b6855d-2915-4943-a742-35629e91fe52","Type":"ContainerDied","Data":"3ebab82011384c1a2f79a9117651ac9daa581a8255e1edcea5a949e40cbc86d5"}
Jul 07 12:49:59.588881 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.588864    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp"
Jul 07 12:49:59.589013 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.588898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp" event={"ID":"e45cc3af-0bb3-4eaa-b798-75e4ff60c012","Type":"ContainerDied","Data":"18e570dd407e0b15a0bea25c1e5150b7c173038533849a214272fa6456fd204e"}
Jul 07 12:49:59.589013 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.588933    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18e570dd407e0b15a0bea25c1e5150b7c173038533849a214272fa6456fd204e"
Jul 07 12:49:59.728998 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.728973    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:49:59.760687 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.760661    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:49:59.865200 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865135    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-util\") pod \"ce07c240-0c91-455e-9fed-3174530ab721\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") "
Jul 07 12:49:59.865200 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865187    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nh25\" (UniqueName: \"kubernetes.io/projected/3d16359c-14b0-49af-a484-16d703780bd2-kube-api-access-9nh25\") pod \"3d16359c-14b0-49af-a484-16d703780bd2\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") "
Jul 07 12:49:59.865371 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865214    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db6ww\" (UniqueName: \"kubernetes.io/projected/ce07c240-0c91-455e-9fed-3174530ab721-kube-api-access-db6ww\") pod \"ce07c240-0c91-455e-9fed-3174530ab721\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") "
Jul 07 12:49:59.865371 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865242    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-bundle\") pod \"3d16359c-14b0-49af-a484-16d703780bd2\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") "
Jul 07 12:49:59.865371 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865276    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-bundle\") pod \"ce07c240-0c91-455e-9fed-3174530ab721\" (UID: \"ce07c240-0c91-455e-9fed-3174530ab721\") "
Jul 07 12:49:59.865371 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865313    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-util\") pod \"3d16359c-14b0-49af-a484-16d703780bd2\" (UID: \"3d16359c-14b0-49af-a484-16d703780bd2\") "
Jul 07 12:49:59.865766 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865740    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-bundle" (OuterVolumeSpecName: "bundle") pod "ce07c240-0c91-455e-9fed-3174530ab721" (UID: "ce07c240-0c91-455e-9fed-3174530ab721"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:49:59.865766 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.865743    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-bundle" (OuterVolumeSpecName: "bundle") pod "3d16359c-14b0-49af-a484-16d703780bd2" (UID: "3d16359c-14b0-49af-a484-16d703780bd2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:49:59.867419 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.867390    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d16359c-14b0-49af-a484-16d703780bd2-kube-api-access-9nh25" (OuterVolumeSpecName: "kube-api-access-9nh25") pod "3d16359c-14b0-49af-a484-16d703780bd2" (UID: "3d16359c-14b0-49af-a484-16d703780bd2"). InnerVolumeSpecName "kube-api-access-9nh25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:49:59.867504 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.867422    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce07c240-0c91-455e-9fed-3174530ab721-kube-api-access-db6ww" (OuterVolumeSpecName: "kube-api-access-db6ww") pod "ce07c240-0c91-455e-9fed-3174530ab721" (UID: "ce07c240-0c91-455e-9fed-3174530ab721"). InnerVolumeSpecName "kube-api-access-db6ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:49:59.870716 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.870683    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-util" (OuterVolumeSpecName: "util") pod "3d16359c-14b0-49af-a484-16d703780bd2" (UID: "3d16359c-14b0-49af-a484-16d703780bd2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:49:59.913870 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.913837    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-util" (OuterVolumeSpecName: "util") pod "ce07c240-0c91-455e-9fed-3174530ab721" (UID: "ce07c240-0c91-455e-9fed-3174530ab721"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:49:59.966291 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.966271    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:59.966291 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.966290    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:59.966394 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.966299    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce07c240-0c91-455e-9fed-3174530ab721-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:59.966394 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.966307    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nh25\" (UniqueName: \"kubernetes.io/projected/3d16359c-14b0-49af-a484-16d703780bd2-kube-api-access-9nh25\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:59.966394 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.966317    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db6ww\" (UniqueName: \"kubernetes.io/projected/ce07c240-0c91-455e-9fed-3174530ab721-kube-api-access-db6ww\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:49:59.966394 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:49:59.966325    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d16359c-14b0-49af-a484-16d703780bd2-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:00.595499 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.595439    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x"
Jul 07 12:50:00.595499 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.595458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x" event={"ID":"3d16359c-14b0-49af-a484-16d703780bd2","Type":"ContainerDied","Data":"08b981ec021a18175fbb4bd48d095f01cef5215ebd59cadad5a873d9216bbe93"}
Jul 07 12:50:00.595873 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.595510    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08b981ec021a18175fbb4bd48d095f01cef5215ebd59cadad5a873d9216bbe93"
Jul 07 12:50:00.597577 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.597550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd" event={"ID":"ce07c240-0c91-455e-9fed-3174530ab721","Type":"ContainerDied","Data":"e763f17f05c595596c1daaea13de4ca84955632868aad383648b94ff8a831ee7"}
Jul 07 12:50:00.597682 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.597580    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd"
Jul 07 12:50:00.597682 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.597597    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e763f17f05c595596c1daaea13de4ca84955632868aad383648b94ff8a831ee7"
Jul 07 12:50:00.723962 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.723943    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:50:00.772484 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.772459    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-bundle\") pod \"26b6855d-2915-4943-a742-35629e91fe52\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") "
Jul 07 12:50:00.772614 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.772497    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45r2t\" (UniqueName: \"kubernetes.io/projected/26b6855d-2915-4943-a742-35629e91fe52-kube-api-access-45r2t\") pod \"26b6855d-2915-4943-a742-35629e91fe52\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") "
Jul 07 12:50:00.772614 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.772548    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-util\") pod \"26b6855d-2915-4943-a742-35629e91fe52\" (UID: \"26b6855d-2915-4943-a742-35629e91fe52\") "
Jul 07 12:50:00.773020 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.772997    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-bundle" (OuterVolumeSpecName: "bundle") pod "26b6855d-2915-4943-a742-35629e91fe52" (UID: "26b6855d-2915-4943-a742-35629e91fe52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:50:00.774443 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.774417    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b6855d-2915-4943-a742-35629e91fe52-kube-api-access-45r2t" (OuterVolumeSpecName: "kube-api-access-45r2t") pod "26b6855d-2915-4943-a742-35629e91fe52" (UID: "26b6855d-2915-4943-a742-35629e91fe52"). InnerVolumeSpecName "kube-api-access-45r2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:50:00.778216 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.778167    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-util" (OuterVolumeSpecName: "util") pod "26b6855d-2915-4943-a742-35629e91fe52" (UID: "26b6855d-2915-4943-a742-35629e91fe52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:50:00.873105 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.873045    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:00.873105 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.873068    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45r2t\" (UniqueName: \"kubernetes.io/projected/26b6855d-2915-4943-a742-35629e91fe52-kube-api-access-45r2t\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:00.873105 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:00.873079    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26b6855d-2915-4943-a742-35629e91fe52-util\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:01.603106 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:01.603061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh" event={"ID":"26b6855d-2915-4943-a742-35629e91fe52","Type":"ContainerDied","Data":"c116530510a6769e39f8c676c360d157d3752b5cfe1880bf7abf73b897d2fbba"}
Jul 07 12:50:01.603473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:01.603119    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c116530510a6769e39f8c676c360d157d3752b5cfe1880bf7abf73b897d2fbba"
Jul 07 12:50:01.603473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:01.603074    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh"
Jul 07 12:50:21.938442 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938405    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"]
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938696    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938708    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938716    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938722    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938729    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938735    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938741    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938746    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938752    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938757    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938765    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938770    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938775    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938781    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938789    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938794    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938801    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938806    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="extract"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938811    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938816    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938823    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938828    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="util"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938836    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="pull"
Jul 07 12:50:21.938839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938840    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="pull"
Jul 07 12:50:21.939510 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938883    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d16359c-14b0-49af-a484-16d703780bd2" containerName="extract"
Jul 07 12:50:21.939510 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938891    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b6855d-2915-4943-a742-35629e91fe52" containerName="extract"
Jul 07 12:50:21.939510 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938897    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce07c240-0c91-455e-9fed-3174530ab721" containerName="extract"
Jul 07 12:50:21.939510 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.938903    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45cc3af-0bb3-4eaa-b798-75e4ff60c012" containerName="extract"
Jul 07 12:50:21.943316 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.943299    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:21.946091 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.946069    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 12:50:21.946211 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.946109    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 12:50:21.946540 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.946476    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 12:50:21.946676 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.946541    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 12:50:21.946856 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.946840    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qlvmx\""
Jul 07 12:50:21.953561 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:21.953541    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"]
Jul 07 12:50:22.026891 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.026860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/52abd8ae-443c-4c01-9918-baa16a0e14a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.026891 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.026894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52abd8ae-443c-4c01-9918-baa16a0e14a7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.027090 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.026927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4s8\" (UniqueName: \"kubernetes.io/projected/52abd8ae-443c-4c01-9918-baa16a0e14a7-kube-api-access-9h4s8\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.127363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.127325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/52abd8ae-443c-4c01-9918-baa16a0e14a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.127363 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.127367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52abd8ae-443c-4c01-9918-baa16a0e14a7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.127658 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.127393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4s8\" (UniqueName: \"kubernetes.io/projected/52abd8ae-443c-4c01-9918-baa16a0e14a7-kube-api-access-9h4s8\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.128043 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.128024    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52abd8ae-443c-4c01-9918-baa16a0e14a7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.129745 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.129721    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/52abd8ae-443c-4c01-9918-baa16a0e14a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.136431 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.136404    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4s8\" (UniqueName: \"kubernetes.io/projected/52abd8ae-443c-4c01-9918-baa16a0e14a7-kube-api-access-9h4s8\") pod \"kuadrant-console-plugin-66b5774985-92rtf\" (UID: \"52abd8ae-443c-4c01-9918-baa16a0e14a7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.252366 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.252283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"
Jul 07 12:50:22.389796 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.389772    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf"]
Jul 07 12:50:22.391978 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:50:22.391945    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52abd8ae_443c_4c01_9918_baa16a0e14a7.slice/crio-54e0d1ea27aa657f101bc4cbd4ff591f485be2d091794b6c5413bf3214c579c1 WatchSource:0}: Error finding container 54e0d1ea27aa657f101bc4cbd4ff591f485be2d091794b6c5413bf3214c579c1: Status 404 returned error can't find the container with id 54e0d1ea27aa657f101bc4cbd4ff591f485be2d091794b6c5413bf3214c579c1
Jul 07 12:50:22.707573 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:22.707538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf" event={"ID":"52abd8ae-443c-4c01-9918-baa16a0e14a7","Type":"ContainerStarted","Data":"54e0d1ea27aa657f101bc4cbd4ff591f485be2d091794b6c5413bf3214c579c1"}
Jul 07 12:50:23.581168 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:23.581132    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc4896648-w4n64"]
Jul 07 12:50:45.832057 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:45.832021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf" event={"ID":"52abd8ae-443c-4c01-9918-baa16a0e14a7","Type":"ContainerStarted","Data":"4c52b005e82df47c4e4db609bd84cd7402e041853a859ee9cf6a38e21c675898"}
Jul 07 12:50:45.847394 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:45.847352    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-92rtf" podStartSLOduration=2.077069865 podStartE2EDuration="24.847339587s" totalImagesPullingTime="22.770269722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:50:45.847273614 +0000 UTC m=+513.848140545" watchObservedRunningTime="2026-07-07 12:50:45.847339587 +0000 UTC m=+513.848206505"
Jul 07 12:50:48.601885 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.601842    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7bc4896648-w4n64" podUID="68b2d5dc-b5d8-4205-98bf-be53bf98f58f" containerName="console" containerID="cri-o://7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39" gracePeriod=15
Jul 07 12:50:48.841909 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.841883    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:50:48.845607 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.845581    2579 generic.go:356] "Generic (PLEG): container finished" podID="68b2d5dc-b5d8-4205-98bf-be53bf98f58f" containerID="7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39" exitCode=0
Jul 07 12:50:48.845725 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.845612    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc4896648-w4n64"
Jul 07 12:50:48.845725 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.845662    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc4896648-w4n64" event={"ID":"68b2d5dc-b5d8-4205-98bf-be53bf98f58f","Type":"ContainerDied","Data":"7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39"}
Jul 07 12:50:48.845725 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.845701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc4896648-w4n64" event={"ID":"68b2d5dc-b5d8-4205-98bf-be53bf98f58f","Type":"ContainerDied","Data":"206b2905be5d4977cb2844d181cbc5f375a764034ba2b5c2c20ac9ae18a224e7"}
Jul 07 12:50:48.845725 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.845720    2579 scope.go:117] "RemoveContainer" containerID="7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39"
Jul 07 12:50:48.853610 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.853533    2579 scope.go:117] "RemoveContainer" containerID="7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39"
Jul 07 12:50:48.853797 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:50:48.853780    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39\": container with ID starting with 7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39 not found: ID does not exist" containerID="7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39"
Jul 07 12:50:48.853839 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.853807    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39"} err="failed to get container status \"7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39\": rpc error: code = NotFound desc = could not find container \"7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39\": container with ID starting with 7c8dd6fef55416a367b65220f0b817db5ddf54aefe2a90075196021627ba0a39 not found: ID does not exist"
Jul 07 12:50:48.946998 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.946965    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-config\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947005    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-serving-cert\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947041    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-oauth-serving-cert\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947069    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-service-ca\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947094    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-trusted-ca-bundle\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947157 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947123    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckwgw\" (UniqueName: \"kubernetes.io/projected/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-kube-api-access-ckwgw\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947406 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947218    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-oauth-config\") pod \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\" (UID: \"68b2d5dc-b5d8-4205-98bf-be53bf98f58f\") "
Jul 07 12:50:48.947496 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947472    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-config" (OuterVolumeSpecName: "console-config") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:50:48.947611 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947589    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-service-ca" (OuterVolumeSpecName: "service-ca") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:50:48.947670 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947604    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:50:48.947771 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.947747    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:50:48.949259 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.949232    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:50:48.949401 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.949278    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:50:48.949401 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:48.949364    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-kube-api-access-ckwgw" (OuterVolumeSpecName: "kube-api-access-ckwgw") pod "68b2d5dc-b5d8-4205-98bf-be53bf98f58f" (UID: "68b2d5dc-b5d8-4205-98bf-be53bf98f58f"). InnerVolumeSpecName "kube-api-access-ckwgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:50:49.048794 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048757    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-config\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.048794 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048782    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-serving-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.048794 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048796    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-oauth-serving-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.049024 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048809    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.049024 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048822    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-trusted-ca-bundle\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.049024 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048841    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckwgw\" (UniqueName: \"kubernetes.io/projected/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-kube-api-access-ckwgw\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.049024 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.048854    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68b2d5dc-b5d8-4205-98bf-be53bf98f58f-console-oauth-config\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:50:49.169772 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.169741    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc4896648-w4n64"]
Jul 07 12:50:49.173673 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:49.173650    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bc4896648-w4n64"]
Jul 07 12:50:50.517505 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:50:50.517474    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68b2d5dc-b5d8-4205-98bf-be53bf98f58f" path="/var/lib/kubelet/pods/68b2d5dc-b5d8-4205-98bf-be53bf98f58f/volumes"
Jul 07 12:51:06.124125 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.124049    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:51:06.124594 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.124365    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68b2d5dc-b5d8-4205-98bf-be53bf98f58f" containerName="console"
Jul 07 12:51:06.124594 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.124379    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="68b2d5dc-b5d8-4205-98bf-be53bf98f58f" containerName="console"
Jul 07 12:51:06.124594 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.124441    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="68b2d5dc-b5d8-4205-98bf-be53bf98f58f" containerName="console"
Jul 07 12:51:06.135866 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.135842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.138380 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.138351    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 12:51:06.140705 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.140684    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:51:06.169147 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.169128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e732c66-ea49-4b2e-a0d0-4a2e44e52d55-config-file\") pod \"limitador-limitador-54d6d988d5-sqzl2\" (UID: \"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.169250 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.169162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt86x\" (UniqueName: \"kubernetes.io/projected/2e732c66-ea49-4b2e-a0d0-4a2e44e52d55-kube-api-access-rt86x\") pod \"limitador-limitador-54d6d988d5-sqzl2\" (UID: \"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.178539 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.178504    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:51:06.270063 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.270035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e732c66-ea49-4b2e-a0d0-4a2e44e52d55-config-file\") pod \"limitador-limitador-54d6d988d5-sqzl2\" (UID: \"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.270206 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.270073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt86x\" (UniqueName: \"kubernetes.io/projected/2e732c66-ea49-4b2e-a0d0-4a2e44e52d55-kube-api-access-rt86x\") pod \"limitador-limitador-54d6d988d5-sqzl2\" (UID: \"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.270690 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.270672    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e732c66-ea49-4b2e-a0d0-4a2e44e52d55-config-file\") pod \"limitador-limitador-54d6d988d5-sqzl2\" (UID: \"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.278842 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.278820    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt86x\" (UniqueName: \"kubernetes.io/projected/2e732c66-ea49-4b2e-a0d0-4a2e44e52d55-kube-api-access-rt86x\") pod \"limitador-limitador-54d6d988d5-sqzl2\" (UID: \"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.446914 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.446844    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:06.586502 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.586465    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xjxld"]
Jul 07 12:51:06.595161 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.595137    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:51:06.595288 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.595267    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:06.595581 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:51:06.595554    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e732c66_ea49_4b2e_a0d0_4a2e44e52d55.slice/crio-391ba594b7ac5189d142d9f4e1a4acda27caa1d3c5e304d94ac395244e213a0d WatchSource:0}: Error finding container 391ba594b7ac5189d142d9f4e1a4acda27caa1d3c5e304d94ac395244e213a0d: Status 404 returned error can't find the container with id 391ba594b7ac5189d142d9f4e1a4acda27caa1d3c5e304d94ac395244e213a0d
Jul 07 12:51:06.597492 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.597388    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5jxwj\""
Jul 07 12:51:06.598610 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.598592    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xjxld"]
Jul 07 12:51:06.673206 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.673174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c79m5\" (UniqueName: \"kubernetes.io/projected/877e7535-7b00-4b1f-bbea-3fc56fb0e599-kube-api-access-c79m5\") pod \"authorino-58cd955675-xjxld\" (UID: \"877e7535-7b00-4b1f-bbea-3fc56fb0e599\") " pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:06.773833 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.773769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c79m5\" (UniqueName: \"kubernetes.io/projected/877e7535-7b00-4b1f-bbea-3fc56fb0e599-kube-api-access-c79m5\") pod \"authorino-58cd955675-xjxld\" (UID: \"877e7535-7b00-4b1f-bbea-3fc56fb0e599\") " pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:06.783146 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.783117    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c79m5\" (UniqueName: \"kubernetes.io/projected/877e7535-7b00-4b1f-bbea-3fc56fb0e599-kube-api-access-c79m5\") pod \"authorino-58cd955675-xjxld\" (UID: \"877e7535-7b00-4b1f-bbea-3fc56fb0e599\") " pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:06.911684 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.911654    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:06.928903 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:06.928875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2" event={"ID":"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55","Type":"ContainerStarted","Data":"391ba594b7ac5189d142d9f4e1a4acda27caa1d3c5e304d94ac395244e213a0d"}
Jul 07 12:51:07.051216 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:07.051152    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xjxld"]
Jul 07 12:51:07.053883 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:51:07.053857    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod877e7535_7b00_4b1f_bbea_3fc56fb0e599.slice/crio-cb43f2b286639fd934df3c7ce97a2addfa4cd6f47a559f3a485518ebfdeaa1f7 WatchSource:0}: Error finding container cb43f2b286639fd934df3c7ce97a2addfa4cd6f47a559f3a485518ebfdeaa1f7: Status 404 returned error can't find the container with id cb43f2b286639fd934df3c7ce97a2addfa4cd6f47a559f3a485518ebfdeaa1f7
Jul 07 12:51:07.936818 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:07.936779    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xjxld" event={"ID":"877e7535-7b00-4b1f-bbea-3fc56fb0e599","Type":"ContainerStarted","Data":"cb43f2b286639fd934df3c7ce97a2addfa4cd6f47a559f3a485518ebfdeaa1f7"}
Jul 07 12:51:11.838082 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:11.838047    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xjxld"]
Jul 07 12:51:11.962500 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:11.962421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2" event={"ID":"2e732c66-ea49-4b2e-a0d0-4a2e44e52d55","Type":"ContainerStarted","Data":"8713fe3c46eaaeabeeb86373195e86481314477cfc94d4d4de89d7fb17cfa836"}
Jul 07 12:51:11.962665 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:11.962535    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:11.963828 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:11.963803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xjxld" event={"ID":"877e7535-7b00-4b1f-bbea-3fc56fb0e599","Type":"ContainerStarted","Data":"68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017"}
Jul 07 12:51:11.977761 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:11.977716    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2" podStartSLOduration=0.901507677 podStartE2EDuration="5.977701529s" totalImagesPullingTime="5.076193852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:11.976583597 +0000 UTC m=+539.977450514" watchObservedRunningTime="2026-07-07 12:51:11.977701529 +0000 UTC m=+539.978568446"
Jul 07 12:51:11.989424 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:11.989385    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xjxld" podStartSLOduration=1.369569007 podStartE2EDuration="5.989371958s" totalImagesPullingTime="4.619802951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:11.988126554 +0000 UTC m=+539.988993471" watchObservedRunningTime="2026-07-07 12:51:11.989371958 +0000 UTC m=+539.990238904"
Jul 07 12:51:12.968401 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:12.968367    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xjxld" podUID="877e7535-7b00-4b1f-bbea-3fc56fb0e599" containerName="authorino" containerID="cri-o://68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017" gracePeriod=30
Jul 07 12:51:13.207718 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.207696    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:13.329250 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.329226    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c79m5\" (UniqueName: \"kubernetes.io/projected/877e7535-7b00-4b1f-bbea-3fc56fb0e599-kube-api-access-c79m5\") pod \"877e7535-7b00-4b1f-bbea-3fc56fb0e599\" (UID: \"877e7535-7b00-4b1f-bbea-3fc56fb0e599\") "
Jul 07 12:51:13.331210 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.331181    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877e7535-7b00-4b1f-bbea-3fc56fb0e599-kube-api-access-c79m5" (OuterVolumeSpecName: "kube-api-access-c79m5") pod "877e7535-7b00-4b1f-bbea-3fc56fb0e599" (UID: "877e7535-7b00-4b1f-bbea-3fc56fb0e599"). InnerVolumeSpecName "kube-api-access-c79m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:51:13.429703 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.429681    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c79m5\" (UniqueName: \"kubernetes.io/projected/877e7535-7b00-4b1f-bbea-3fc56fb0e599-kube-api-access-c79m5\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:51:13.973473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.973437    2579 generic.go:356] "Generic (PLEG): container finished" podID="877e7535-7b00-4b1f-bbea-3fc56fb0e599" containerID="68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017" exitCode=0
Jul 07 12:51:13.973883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.973478    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xjxld"
Jul 07 12:51:13.973883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.973534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xjxld" event={"ID":"877e7535-7b00-4b1f-bbea-3fc56fb0e599","Type":"ContainerDied","Data":"68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017"}
Jul 07 12:51:13.973883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.973573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xjxld" event={"ID":"877e7535-7b00-4b1f-bbea-3fc56fb0e599","Type":"ContainerDied","Data":"cb43f2b286639fd934df3c7ce97a2addfa4cd6f47a559f3a485518ebfdeaa1f7"}
Jul 07 12:51:13.973883 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.973589    2579 scope.go:117] "RemoveContainer" containerID="68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017"
Jul 07 12:51:13.981905 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.981892    2579 scope.go:117] "RemoveContainer" containerID="68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017"
Jul 07 12:51:13.982135 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:51:13.982115    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017\": container with ID starting with 68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017 not found: ID does not exist" containerID="68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017"
Jul 07 12:51:13.982180 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.982143    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017"} err="failed to get container status \"68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017\": rpc error: code = NotFound desc = could not find container \"68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017\": container with ID starting with 68cfc5100ab8f562b64931c8856a5ebd81f3c3c2c8b3939807ab15293ca23017 not found: ID does not exist"
Jul 07 12:51:13.992173 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.992149    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xjxld"]
Jul 07 12:51:13.994134 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:13.994116    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xjxld"]
Jul 07 12:51:14.517432 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:14.517401    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877e7535-7b00-4b1f-bbea-3fc56fb0e599" path="/var/lib/kubelet/pods/877e7535-7b00-4b1f-bbea-3fc56fb0e599/volumes"
Jul 07 12:51:22.969725 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:22.969687    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"
Jul 07 12:51:39.524557 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.524512    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76shk"]
Jul 07 12:51:39.524954 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.524821    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877e7535-7b00-4b1f-bbea-3fc56fb0e599" containerName="authorino"
Jul 07 12:51:39.524954 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.524834    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="877e7535-7b00-4b1f-bbea-3fc56fb0e599" containerName="authorino"
Jul 07 12:51:39.524954 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.524905    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="877e7535-7b00-4b1f-bbea-3fc56fb0e599" containerName="authorino"
Jul 07 12:51:39.526576 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.526562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:39.530750 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.530728    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5jxwj\""
Jul 07 12:51:39.537447 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.537425    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76shk"]
Jul 07 12:51:39.620728 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.620693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndsgr\" (UniqueName: \"kubernetes.io/projected/19314150-1179-4b65-ae34-ce91fbf9a82e-kube-api-access-ndsgr\") pod \"authorino-5d4b67c446-76shk\" (UID: \"19314150-1179-4b65-ae34-ce91fbf9a82e\") " pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:39.721473 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.721436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndsgr\" (UniqueName: \"kubernetes.io/projected/19314150-1179-4b65-ae34-ce91fbf9a82e-kube-api-access-ndsgr\") pod \"authorino-5d4b67c446-76shk\" (UID: \"19314150-1179-4b65-ae34-ce91fbf9a82e\") " pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:39.730824 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.730800    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndsgr\" (UniqueName: \"kubernetes.io/projected/19314150-1179-4b65-ae34-ce91fbf9a82e-kube-api-access-ndsgr\") pod \"authorino-5d4b67c446-76shk\" (UID: \"19314150-1179-4b65-ae34-ce91fbf9a82e\") " pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:39.775699 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.775618    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76shk"]
Jul 07 12:51:39.775848 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.775836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:39.806658 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.806628    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d9f9bcf54-7gkms"]
Jul 07 12:51:39.808814 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.808793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:39.819425 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.819402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d9f9bcf54-7gkms"]
Jul 07 12:51:39.905355 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.905326    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76shk"]
Jul 07 12:51:39.907197 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:51:39.907169    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19314150_1179_4b65_ae34_ce91fbf9a82e.slice/crio-6ffe7e76bcfc66de77b8653ce39b70bbc3b75de084aaeb58faa6d1d34f345650 WatchSource:0}: Error finding container 6ffe7e76bcfc66de77b8653ce39b70bbc3b75de084aaeb58faa6d1d34f345650: Status 404 returned error can't find the container with id 6ffe7e76bcfc66de77b8653ce39b70bbc3b75de084aaeb58faa6d1d34f345650
Jul 07 12:51:39.923433 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:39.923410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72w5s\" (UniqueName: \"kubernetes.io/projected/51d43415-9df7-470c-a51c-674b39039e63-kube-api-access-72w5s\") pod \"authorino-5d9f9bcf54-7gkms\" (UID: \"51d43415-9df7-470c-a51c-674b39039e63\") " pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:40.023948 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.023918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72w5s\" (UniqueName: \"kubernetes.io/projected/51d43415-9df7-470c-a51c-674b39039e63-kube-api-access-72w5s\") pod \"authorino-5d9f9bcf54-7gkms\" (UID: \"51d43415-9df7-470c-a51c-674b39039e63\") " pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:40.032633 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.032586    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72w5s\" (UniqueName: \"kubernetes.io/projected/51d43415-9df7-470c-a51c-674b39039e63-kube-api-access-72w5s\") pod \"authorino-5d9f9bcf54-7gkms\" (UID: \"51d43415-9df7-470c-a51c-674b39039e63\") " pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:40.096154 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.096124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76shk" event={"ID":"19314150-1179-4b65-ae34-ce91fbf9a82e","Type":"ContainerStarted","Data":"6ffe7e76bcfc66de77b8653ce39b70bbc3b75de084aaeb58faa6d1d34f345650"}
Jul 07 12:51:40.119266 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.119235    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:40.134408 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.134383    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d9f9bcf54-7gkms"]
Jul 07 12:51:40.163649 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.163619    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7445cd9486-kj8jq"]
Jul 07 12:51:40.166159 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.166104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.168188 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.168166    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 12:51:40.175171 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.175109    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7445cd9486-kj8jq"]
Jul 07 12:51:40.251225 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.251202    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d9f9bcf54-7gkms"]
Jul 07 12:51:40.252771 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:51:40.252747    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51d43415_9df7_470c_a51c_674b39039e63.slice/crio-6eaa77ca1bc12586be22ef02818648e873a66bba59320f03c213f7acaad47830 WatchSource:0}: Error finding container 6eaa77ca1bc12586be22ef02818648e873a66bba59320f03c213f7acaad47830: Status 404 returned error can't find the container with id 6eaa77ca1bc12586be22ef02818648e873a66bba59320f03c213f7acaad47830
Jul 07 12:51:40.326050 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.326020    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-tls-cert\") pod \"authorino-7445cd9486-kj8jq\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") " pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.326209 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.326086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pm8g\" (UniqueName: \"kubernetes.io/projected/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-kube-api-access-4pm8g\") pod \"authorino-7445cd9486-kj8jq\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") " pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.426590 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.426549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pm8g\" (UniqueName: \"kubernetes.io/projected/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-kube-api-access-4pm8g\") pod \"authorino-7445cd9486-kj8jq\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") " pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.426755 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.426604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-tls-cert\") pod \"authorino-7445cd9486-kj8jq\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") " pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.428974 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.428947    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-tls-cert\") pod \"authorino-7445cd9486-kj8jq\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") " pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.434392 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.434374    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pm8g\" (UniqueName: \"kubernetes.io/projected/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-kube-api-access-4pm8g\") pod \"authorino-7445cd9486-kj8jq\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") " pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.478449 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.478422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:51:40.616935 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:40.616874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7445cd9486-kj8jq"]
Jul 07 12:51:40.619236 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:51:40.619205    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff47cd88_7f0a_41d3_b715_2bccc45f2a73.slice/crio-47197980226ddfc0ff674ebe57a53175aa910759e69f433bf077dd9f1acf4fed WatchSource:0}: Error finding container 47197980226ddfc0ff674ebe57a53175aa910759e69f433bf077dd9f1acf4fed: Status 404 returned error can't find the container with id 47197980226ddfc0ff674ebe57a53175aa910759e69f433bf077dd9f1acf4fed
Jul 07 12:51:41.102828 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.102773    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-76shk" podUID="19314150-1179-4b65-ae34-ce91fbf9a82e" containerName="authorino" containerID="cri-o://63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6" gracePeriod=30
Jul 07 12:51:41.102828 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.102809    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76shk" event={"ID":"19314150-1179-4b65-ae34-ce91fbf9a82e","Type":"ContainerStarted","Data":"63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6"}
Jul 07 12:51:41.104595 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.104568    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms" event={"ID":"51d43415-9df7-470c-a51c-674b39039e63","Type":"ContainerStarted","Data":"3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06"}
Jul 07 12:51:41.104743 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.104605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms" event={"ID":"51d43415-9df7-470c-a51c-674b39039e63","Type":"ContainerStarted","Data":"6eaa77ca1bc12586be22ef02818648e873a66bba59320f03c213f7acaad47830"}
Jul 07 12:51:41.104743 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.104600    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms" podUID="51d43415-9df7-470c-a51c-674b39039e63" containerName="authorino" containerID="cri-o://3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06" gracePeriod=30
Jul 07 12:51:41.106404 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.106375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7445cd9486-kj8jq" event={"ID":"ff47cd88-7f0a-41d3-b715-2bccc45f2a73","Type":"ContainerStarted","Data":"fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58"}
Jul 07 12:51:41.106535 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.106414    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7445cd9486-kj8jq" event={"ID":"ff47cd88-7f0a-41d3-b715-2bccc45f2a73","Type":"ContainerStarted","Data":"47197980226ddfc0ff674ebe57a53175aa910759e69f433bf077dd9f1acf4fed"}
Jul 07 12:51:41.119581 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.119540    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-76shk" podStartSLOduration=1.5147351740000001 podStartE2EDuration="2.119500604s" totalImagesPullingTime="604.76543ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:41.117540733 +0000 UTC m=+569.118407653" watchObservedRunningTime="2026-07-07 12:51:41.119500604 +0000 UTC m=+569.120367556"
Jul 07 12:51:41.133985 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.133910    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7445cd9486-kj8jq" podStartSLOduration=0.747124678 podStartE2EDuration="1.133863436s" totalImagesPullingTime="386.738758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:41.132179142 +0000 UTC m=+569.133046062" watchObservedRunningTime="2026-07-07 12:51:41.133863436 +0000 UTC m=+569.134730353"
Jul 07 12:51:41.151989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.151820    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms" podStartSLOduration=1.780611842 podStartE2EDuration="2.151800333s" totalImagesPullingTime="371.188491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:51:41.151440352 +0000 UTC m=+569.152307267" watchObservedRunningTime="2026-07-07 12:51:41.151800333 +0000 UTC m=+569.152667252"
Jul 07 12:51:41.352976 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.352951    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:41.359919 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.359902    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:41.433984 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.433952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndsgr\" (UniqueName: \"kubernetes.io/projected/19314150-1179-4b65-ae34-ce91fbf9a82e-kube-api-access-ndsgr\") pod \"19314150-1179-4b65-ae34-ce91fbf9a82e\" (UID: \"19314150-1179-4b65-ae34-ce91fbf9a82e\") "
Jul 07 12:51:41.435918 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.435896    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19314150-1179-4b65-ae34-ce91fbf9a82e-kube-api-access-ndsgr" (OuterVolumeSpecName: "kube-api-access-ndsgr") pod "19314150-1179-4b65-ae34-ce91fbf9a82e" (UID: "19314150-1179-4b65-ae34-ce91fbf9a82e"). InnerVolumeSpecName "kube-api-access-ndsgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:51:41.535375 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.535296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72w5s\" (UniqueName: \"kubernetes.io/projected/51d43415-9df7-470c-a51c-674b39039e63-kube-api-access-72w5s\") pod \"51d43415-9df7-470c-a51c-674b39039e63\" (UID: \"51d43415-9df7-470c-a51c-674b39039e63\") "
Jul 07 12:51:41.535513 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.535446    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndsgr\" (UniqueName: \"kubernetes.io/projected/19314150-1179-4b65-ae34-ce91fbf9a82e-kube-api-access-ndsgr\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:51:41.537279 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.537254    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d43415-9df7-470c-a51c-674b39039e63-kube-api-access-72w5s" (OuterVolumeSpecName: "kube-api-access-72w5s") pod "51d43415-9df7-470c-a51c-674b39039e63" (UID: "51d43415-9df7-470c-a51c-674b39039e63"). InnerVolumeSpecName "kube-api-access-72w5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:51:41.636808 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:41.636775    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72w5s\" (UniqueName: \"kubernetes.io/projected/51d43415-9df7-470c-a51c-674b39039e63-kube-api-access-72w5s\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:51:42.113497 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.113459    2579 generic.go:356] "Generic (PLEG): container finished" podID="51d43415-9df7-470c-a51c-674b39039e63" containerID="3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06" exitCode=0
Jul 07 12:51:42.113717 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.113504    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms" event={"ID":"51d43415-9df7-470c-a51c-674b39039e63","Type":"ContainerDied","Data":"3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06"}
Jul 07 12:51:42.113717 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.113574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms" event={"ID":"51d43415-9df7-470c-a51c-674b39039e63","Type":"ContainerDied","Data":"6eaa77ca1bc12586be22ef02818648e873a66bba59320f03c213f7acaad47830"}
Jul 07 12:51:42.113717 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.113597    2579 scope.go:117] "RemoveContainer" containerID="3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06"
Jul 07 12:51:42.113717 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.113511    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d9f9bcf54-7gkms"
Jul 07 12:51:42.115205 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.115172    2579 generic.go:356] "Generic (PLEG): container finished" podID="19314150-1179-4b65-ae34-ce91fbf9a82e" containerID="63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6" exitCode=0
Jul 07 12:51:42.115304 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.115213    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76shk"
Jul 07 12:51:42.115304 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.115252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76shk" event={"ID":"19314150-1179-4b65-ae34-ce91fbf9a82e","Type":"ContainerDied","Data":"63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6"}
Jul 07 12:51:42.115304 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.115292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76shk" event={"ID":"19314150-1179-4b65-ae34-ce91fbf9a82e","Type":"ContainerDied","Data":"6ffe7e76bcfc66de77b8653ce39b70bbc3b75de084aaeb58faa6d1d34f345650"}
Jul 07 12:51:42.123893 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.123873    2579 scope.go:117] "RemoveContainer" containerID="3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06"
Jul 07 12:51:42.124501 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:51:42.124469    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06\": container with ID starting with 3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06 not found: ID does not exist" containerID="3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06"
Jul 07 12:51:42.124655 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.124513    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06"} err="failed to get container status \"3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06\": rpc error: code = NotFound desc = could not find container \"3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06\": container with ID starting with 3fab03ad0438faf5b9715740f04e3dfece7396606dbf33c3a57606d81ac7ad06 not found: ID does not exist"
Jul 07 12:51:42.124655 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.124558    2579 scope.go:117] "RemoveContainer" containerID="63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6"
Jul 07 12:51:42.132815 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.132795    2579 scope.go:117] "RemoveContainer" containerID="63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6"
Jul 07 12:51:42.133148 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:51:42.133127    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6\": container with ID starting with 63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6 not found: ID does not exist" containerID="63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6"
Jul 07 12:51:42.133228 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.133155    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6"} err="failed to get container status \"63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6\": rpc error: code = NotFound desc = could not find container \"63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6\": container with ID starting with 63af714e43be2f5d526ab13e4983ef37fd2352db53db60dcc0eb24d9814e7ad6 not found: ID does not exist"
Jul 07 12:51:42.137684 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.137662    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d9f9bcf54-7gkms"]
Jul 07 12:51:42.141865 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.141841    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d9f9bcf54-7gkms"]
Jul 07 12:51:42.150617 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.150592    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76shk"]
Jul 07 12:51:42.154253 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.154231    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76shk"]
Jul 07 12:51:42.518802 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.518722    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19314150-1179-4b65-ae34-ce91fbf9a82e" path="/var/lib/kubelet/pods/19314150-1179-4b65-ae34-ce91fbf9a82e/volumes"
Jul 07 12:51:42.519037 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:51:42.519025    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d43415-9df7-470c-a51c-674b39039e63" path="/var/lib/kubelet/pods/51d43415-9df7-470c-a51c-674b39039e63/volumes"
Jul 07 12:52:19.780485 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:52:19.780448    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:52:35.069241 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:52:35.069161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:52:50.266774 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:52:50.266737    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:52:57.075417 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:52:57.075383    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:53:01.092236 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:01.092196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:53:11.166970 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:11.166925    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:53:20.971059 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971028    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"]
Jul 07 12:53:20.971464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971299    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d43415-9df7-470c-a51c-674b39039e63" containerName="authorino"
Jul 07 12:53:20.971464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971309    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d43415-9df7-470c-a51c-674b39039e63" containerName="authorino"
Jul 07 12:53:20.971464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971321    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19314150-1179-4b65-ae34-ce91fbf9a82e" containerName="authorino"
Jul 07 12:53:20.971464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971326    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="19314150-1179-4b65-ae34-ce91fbf9a82e" containerName="authorino"
Jul 07 12:53:20.971464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971373    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="19314150-1179-4b65-ae34-ce91fbf9a82e" containerName="authorino"
Jul 07 12:53:20.971464 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.971383    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d43415-9df7-470c-a51c-674b39039e63" containerName="authorino"
Jul 07 12:53:20.974038 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.974022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.976771 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.976746    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 12:53:20.976918 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.976802    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 12:53:20.976918 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.976802    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 12:53:20.976918 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.976896    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f7d92\""
Jul 07 12:53:20.979408 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.979386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.979535 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.979434    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.979535 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.979497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.979675 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.979658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.979734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.979692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.979734 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.979719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6q4q\" (UniqueName: \"kubernetes.io/projected/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-kube-api-access-h6q4q\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:20.988539 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:20.988503    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"]
Jul 07 12:53:21.080205 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080362 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080362 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080362 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080362 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6q4q\" (UniqueName: \"kubernetes.io/projected/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-kube-api-access-h6q4q\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080362 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080667 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080647    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080711 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080691    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.080792 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.080766    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.082594 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.082573    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.082787 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.082770    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.088273 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.088253    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6q4q\" (UniqueName: \"kubernetes.io/projected/2fb4a324-8007-4e86-9c6f-07e9f06a1ee0-kube-api-access-h6q4q\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw\" (UID: \"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.283323 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.283223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:21.412950 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.412929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"]
Jul 07 12:53:21.414904 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:53:21.414879    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fb4a324_8007_4e86_9c6f_07e9f06a1ee0.slice/crio-306ec3e20e7f4bdda8995d1bd7e4a53642de031acd88865f7038b22505701748 WatchSource:0}: Error finding container 306ec3e20e7f4bdda8995d1bd7e4a53642de031acd88865f7038b22505701748: Status 404 returned error can't find the container with id 306ec3e20e7f4bdda8995d1bd7e4a53642de031acd88865f7038b22505701748
Jul 07 12:53:21.416512 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.416494    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 12:53:21.575557 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:21.575506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw" event={"ID":"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0","Type":"ContainerStarted","Data":"306ec3e20e7f4bdda8995d1bd7e4a53642de031acd88865f7038b22505701748"}
Jul 07 12:53:23.275931 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:23.275900    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:53:23.586083 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:23.586049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw" event={"ID":"2fb4a324-8007-4e86-9c6f-07e9f06a1ee0","Type":"ContainerStarted","Data":"b647fd584cc2eccaada179b783ba4ad93a990465aae287a47cb536b5c547c5de"}
Jul 07 12:53:31.284233 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:31.284198    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:31.284233 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:31.284242    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:31.296285 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:31.296258    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:53:31.313118 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:31.313070    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw" podStartSLOduration=10.138324467 podStartE2EDuration="11.313051859s" totalImagesPullingTime="1.174727392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:53:23.603690142 +0000 UTC m=+671.604557062" watchObservedRunningTime="2026-07-07 12:53:31.313051859 +0000 UTC m=+679.313918776"
Jul 07 12:53:31.633099 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:53:31.633071    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw"
Jul 07 12:54:32.998447 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:32.998408    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b68756c4c-4d4zl"]
Jul 07 12:54:33.000680 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.000660    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.014295 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.014272    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b68756c4c-4d4zl"]
Jul 07 12:54:33.049566 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.049536    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpmf7\" (UniqueName: \"kubernetes.io/projected/1af2a399-201c-4105-a1f0-a38ef29d26f4-kube-api-access-mpmf7\") pod \"authorino-6b68756c4c-4d4zl\" (UID: \"1af2a399-201c-4105-a1f0-a38ef29d26f4\") " pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.049726 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.049605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1af2a399-201c-4105-a1f0-a38ef29d26f4-tls-cert\") pod \"authorino-6b68756c4c-4d4zl\" (UID: \"1af2a399-201c-4105-a1f0-a38ef29d26f4\") " pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.150916 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.150886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1af2a399-201c-4105-a1f0-a38ef29d26f4-tls-cert\") pod \"authorino-6b68756c4c-4d4zl\" (UID: \"1af2a399-201c-4105-a1f0-a38ef29d26f4\") " pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.151068 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.150962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpmf7\" (UniqueName: \"kubernetes.io/projected/1af2a399-201c-4105-a1f0-a38ef29d26f4-kube-api-access-mpmf7\") pod \"authorino-6b68756c4c-4d4zl\" (UID: \"1af2a399-201c-4105-a1f0-a38ef29d26f4\") " pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.153259 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.153237    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1af2a399-201c-4105-a1f0-a38ef29d26f4-tls-cert\") pod \"authorino-6b68756c4c-4d4zl\" (UID: \"1af2a399-201c-4105-a1f0-a38ef29d26f4\") " pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.158823 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.158795    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpmf7\" (UniqueName: \"kubernetes.io/projected/1af2a399-201c-4105-a1f0-a38ef29d26f4-kube-api-access-mpmf7\") pod \"authorino-6b68756c4c-4d4zl\" (UID: \"1af2a399-201c-4105-a1f0-a38ef29d26f4\") " pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.309165 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.309086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b68756c4c-4d4zl"
Jul 07 12:54:33.462958 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.462851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b68756c4c-4d4zl"]
Jul 07 12:54:33.465851 ip-10-0-132-141 kubenswrapper[2579]: W0707 12:54:33.465811    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1af2a399_201c_4105_a1f0_a38ef29d26f4.slice/crio-d2c1f28de301b96439041502d446fc707dbc5d8835d4237786a6088b3d23ebf6 WatchSource:0}: Error finding container d2c1f28de301b96439041502d446fc707dbc5d8835d4237786a6088b3d23ebf6: Status 404 returned error can't find the container with id d2c1f28de301b96439041502d446fc707dbc5d8835d4237786a6088b3d23ebf6
Jul 07 12:54:33.911027 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.910948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b68756c4c-4d4zl" event={"ID":"1af2a399-201c-4105-a1f0-a38ef29d26f4","Type":"ContainerStarted","Data":"212dbf97f427d2420024905d452d204145152f302f554eea050033b25abc3c03"}
Jul 07 12:54:33.911027 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.910999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b68756c4c-4d4zl" event={"ID":"1af2a399-201c-4105-a1f0-a38ef29d26f4","Type":"ContainerStarted","Data":"d2c1f28de301b96439041502d446fc707dbc5d8835d4237786a6088b3d23ebf6"}
Jul 07 12:54:33.925947 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.925877    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b68756c4c-4d4zl" podStartSLOduration=1.6244822829999999 podStartE2EDuration="1.92585755s" totalImagesPullingTime="301.375267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:54:33.925004035 +0000 UTC m=+741.925870953" watchObservedRunningTime="2026-07-07 12:54:33.92585755 +0000 UTC m=+741.926724471"
Jul 07 12:54:33.952513 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.952478    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7445cd9486-kj8jq"]
Jul 07 12:54:33.952691 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:33.952671    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7445cd9486-kj8jq" podUID="ff47cd88-7f0a-41d3-b715-2bccc45f2a73" containerName="authorino" containerID="cri-o://fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58" gracePeriod=30
Jul 07 12:54:34.202983 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.202955    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:54:34.261172 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.261143    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-tls-cert\") pod \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") "
Jul 07 12:54:34.261338 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.261196    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pm8g\" (UniqueName: \"kubernetes.io/projected/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-kube-api-access-4pm8g\") pod \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\" (UID: \"ff47cd88-7f0a-41d3-b715-2bccc45f2a73\") "
Jul 07 12:54:34.263547 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.263498    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-kube-api-access-4pm8g" (OuterVolumeSpecName: "kube-api-access-4pm8g") pod "ff47cd88-7f0a-41d3-b715-2bccc45f2a73" (UID: "ff47cd88-7f0a-41d3-b715-2bccc45f2a73"). InnerVolumeSpecName "kube-api-access-4pm8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:54:34.271647 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.271613    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ff47cd88-7f0a-41d3-b715-2bccc45f2a73" (UID: "ff47cd88-7f0a-41d3-b715-2bccc45f2a73"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:54:34.362432 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.362353    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-tls-cert\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:54:34.362432 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.362382    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pm8g\" (UniqueName: \"kubernetes.io/projected/ff47cd88-7f0a-41d3-b715-2bccc45f2a73-kube-api-access-4pm8g\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 12:54:34.917277 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.917237    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff47cd88-7f0a-41d3-b715-2bccc45f2a73" containerID="fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58" exitCode=0
Jul 07 12:54:34.917444 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.917299    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7445cd9486-kj8jq"
Jul 07 12:54:34.917444 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.917323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7445cd9486-kj8jq" event={"ID":"ff47cd88-7f0a-41d3-b715-2bccc45f2a73","Type":"ContainerDied","Data":"fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58"}
Jul 07 12:54:34.917444 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.917374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7445cd9486-kj8jq" event={"ID":"ff47cd88-7f0a-41d3-b715-2bccc45f2a73","Type":"ContainerDied","Data":"47197980226ddfc0ff674ebe57a53175aa910759e69f433bf077dd9f1acf4fed"}
Jul 07 12:54:34.917444 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.917393    2579 scope.go:117] "RemoveContainer" containerID="fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58"
Jul 07 12:54:34.925989 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.925967    2579 scope.go:117] "RemoveContainer" containerID="fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58"
Jul 07 12:54:34.926269 ip-10-0-132-141 kubenswrapper[2579]: E0707 12:54:34.926251    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58\": container with ID starting with fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58 not found: ID does not exist" containerID="fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58"
Jul 07 12:54:34.926366 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.926279    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58"} err="failed to get container status \"fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58\": rpc error: code = NotFound desc = could not find container \"fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58\": container with ID starting with fecfa7909aa6212cf309f4f9c9493e64246948ccb3bdd32bb2068507fe953c58 not found: ID does not exist"
Jul 07 12:54:34.937055 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.937028    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7445cd9486-kj8jq"]
Jul 07 12:54:34.942404 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:34.942378    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7445cd9486-kj8jq"]
Jul 07 12:54:36.517397 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:54:36.517364    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff47cd88-7f0a-41d3-b715-2bccc45f2a73" path="/var/lib/kubelet/pods/ff47cd88-7f0a-41d3-b715-2bccc45f2a73/volumes"
Jul 07 12:55:12.673414 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:55:12.673382    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:55:21.567951 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:55:21.567911    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:55:33.471337 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:55:33.471301    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:55:41.777807 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:55:41.777734    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:55:52.090908 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:55:52.090874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:56:18.284880 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:56:18.284847    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:56:30.470870 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:56:30.470832    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:04.760186 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:04.760095    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:13.464503 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:13.464468    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:22.170030 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:22.169993    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:27.777921 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:27.777880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:39.170779 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:39.170749    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:47.872099 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:47.872062    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:57:59.868490 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:57:59.868448    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:58:09.368923 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:58:09.368884    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:58:32.074103 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:58:32.074070    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:58:40.876887 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:58:40.876816    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:59:05.774539 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:59:05.774489    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:59:15.665662 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:59:15.665627    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:59:29.975368 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:59:29.975336    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:59:39.671047 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:59:39.671008    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:59:50.073682 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:59:50.073645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 12:59:59.172213 ip-10-0-132-141 kubenswrapper[2579]: I0707 12:59:59.172172    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:00:06.969556 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:00:06.969468    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:01:57.077196 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:01:57.077119    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:02:05.876582 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:02:05.876551    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:02:19.979364 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:02:19.979329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:02:31.317107 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:02:31.317079    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:02:42.178884 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:02:42.178849    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:02:52.167870 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:02:52.167834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:03:01.675627 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:03:01.675591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:03:12.868662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:03:12.868629    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:03:23.968828 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:03:23.968794    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:03:32.184439 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:03:32.184406    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:03:45.967792 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:03:45.967746    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:04:02.870986 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:04:02.870947    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:04:12.201280 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:04:12.201245    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:04:23.091551 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:04:23.091097    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:04:35.006005 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:04:35.005918    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:04:50.898898 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:04:50.898864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:00.261292 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:00.261256    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:09.010337 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:09.010309    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:19.269965 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:19.269934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:28.567833 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:28.567797    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:37.882662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:37.882627    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:50.967377 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:50.967341    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:05:59.968948 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:05:59.968911    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:06:09.279808 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:06:09.279758    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:06:20.964805 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:06:20.964773    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:06:31.974566 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:06:31.974536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:06:39.671390 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:06:39.671360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:06:51.383479 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:06:51.383447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:07:02.964092 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:02.964056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:07:13.463459 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:13.463405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:07:25.530578 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.530543    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"]
Jul 07 13:07:25.531032 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.530961    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff47cd88-7f0a-41d3-b715-2bccc45f2a73" containerName="authorino"
Jul 07 13:07:25.531032 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.530978    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff47cd88-7f0a-41d3-b715-2bccc45f2a73" containerName="authorino"
Jul 07 13:07:25.531133 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.531052    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff47cd88-7f0a-41d3-b715-2bccc45f2a73" containerName="authorino"
Jul 07 13:07:25.535229 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.535210    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.537490 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.537461    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-abdf58a0-serving-cert\""
Jul 07 13:07:25.537490 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.537480    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:07:25.537672 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.537499    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:07:25.538010 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.537993    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:07:25.543839 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.543817    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"]
Jul 07 13:07:25.602097 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.602072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.602230 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.602139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhgmx\" (UniqueName: \"kubernetes.io/projected/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-kube-api-access-hhgmx\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.602300 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.602236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-openshift-service-ca\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.703055 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.703030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhgmx\" (UniqueName: \"kubernetes.io/projected/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-kube-api-access-hhgmx\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.703166 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.703067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-openshift-service-ca\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.703166 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.703106    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.703250 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:07:25.703189    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-serving-cert: secret "maas-api-e2e-ait-abdf58a0-serving-cert" not found
Jul 07 13:07:25.703250 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:07:25.703232    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls podName:5d0641e4-fba6-48df-9ef2-3b9685cdedc2 nodeName:}" failed. No retries permitted until 2026-07-07 13:07:26.203217086 +0000 UTC m=+1514.204083983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls") pod "maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" (UID: "5d0641e4-fba6-48df-9ef2-3b9685cdedc2") : secret "maas-api-e2e-ait-abdf58a0-serving-cert" not found
Jul 07 13:07:25.703911 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.703890    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-openshift-service-ca\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:25.711804 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:25.711781    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhgmx\" (UniqueName: \"kubernetes.io/projected/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-kube-api-access-hhgmx\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:26.207901 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:26.207798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:26.210461 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:26.210423    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls\") pod \"maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:26.446387 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:26.446357    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:26.590038 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:26.590011    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"]
Jul 07 13:07:26.592092 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:07:26.592049    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0641e4_fba6_48df_9ef2_3b9685cdedc2.slice/crio-900c8fe9a389dfce717debec6dd6149b9e5877d04dd85c3b0f58418617993da9 WatchSource:0}: Error finding container 900c8fe9a389dfce717debec6dd6149b9e5877d04dd85c3b0f58418617993da9: Status 404 returned error can't find the container with id 900c8fe9a389dfce717debec6dd6149b9e5877d04dd85c3b0f58418617993da9
Jul 07 13:07:26.593479 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:26.593458    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:07:26.641418 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:26.641388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" event={"ID":"5d0641e4-fba6-48df-9ef2-3b9685cdedc2","Type":"ContainerStarted","Data":"900c8fe9a389dfce717debec6dd6149b9e5877d04dd85c3b0f58418617993da9"}
Jul 07 13:07:28.654508 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:28.654469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" event={"ID":"5d0641e4-fba6-48df-9ef2-3b9685cdedc2","Type":"ContainerStarted","Data":"0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4"}
Jul 07 13:07:28.654924 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:28.654626    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:28.672507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:28.672460    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" podStartSLOduration=1.94511629 podStartE2EDuration="3.672441597s" totalImagesPullingTime="1.727325307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:07:28.669846513 +0000 UTC m=+1516.670713431" watchObservedRunningTime="2026-07-07 13:07:28.672441597 +0000 UTC m=+1516.673308515"
Jul 07 13:07:29.379268 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:29.377543    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:07:30.852606 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:30.852561    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"]
Jul 07 13:07:30.853067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:30.852819    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" podUID="5d0641e4-fba6-48df-9ef2-3b9685cdedc2" containerName="maas-api" containerID="cri-o://0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4" gracePeriod=30
Jul 07 13:07:31.097939 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.097919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:31.143761 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.143705    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-openshift-service-ca\") pod \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") "
Jul 07 13:07:31.143761 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.143742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls\") pod \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") "
Jul 07 13:07:31.143884 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.143762    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhgmx\" (UniqueName: \"kubernetes.io/projected/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-kube-api-access-hhgmx\") pod \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\" (UID: \"5d0641e4-fba6-48df-9ef2-3b9685cdedc2\") "
Jul 07 13:07:31.144103 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.144069    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d0641e4-fba6-48df-9ef2-3b9685cdedc2" (UID: "5d0641e4-fba6-48df-9ef2-3b9685cdedc2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:07:31.145790 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.145764    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-kube-api-access-hhgmx" (OuterVolumeSpecName: "kube-api-access-hhgmx") pod "5d0641e4-fba6-48df-9ef2-3b9685cdedc2" (UID: "5d0641e4-fba6-48df-9ef2-3b9685cdedc2"). InnerVolumeSpecName "kube-api-access-hhgmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:07:31.145905 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.145813    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d0641e4-fba6-48df-9ef2-3b9685cdedc2" (UID: "5d0641e4-fba6-48df-9ef2-3b9685cdedc2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:07:31.245234 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.245212    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:07:31.245234 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.245231    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:07:31.245359 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.245240    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhgmx\" (UniqueName: \"kubernetes.io/projected/5d0641e4-fba6-48df-9ef2-3b9685cdedc2-kube-api-access-hhgmx\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:07:31.670501 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.670467    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d0641e4-fba6-48df-9ef2-3b9685cdedc2" containerID="0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4" exitCode=0
Jul 07 13:07:31.672398 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.670532    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"
Jul 07 13:07:31.672398 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.670509    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" event={"ID":"5d0641e4-fba6-48df-9ef2-3b9685cdedc2","Type":"ContainerDied","Data":"0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4"}
Jul 07 13:07:31.672398 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.670590    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj" event={"ID":"5d0641e4-fba6-48df-9ef2-3b9685cdedc2","Type":"ContainerDied","Data":"900c8fe9a389dfce717debec6dd6149b9e5877d04dd85c3b0f58418617993da9"}
Jul 07 13:07:31.672398 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.670621    2579 scope.go:117] "RemoveContainer" containerID="0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4"
Jul 07 13:07:31.683855 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.683837    2579 scope.go:117] "RemoveContainer" containerID="0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4"
Jul 07 13:07:31.684132 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:07:31.684101    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4\": container with ID starting with 0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4 not found: ID does not exist" containerID="0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4"
Jul 07 13:07:31.684231 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.684140    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4"} err="failed to get container status \"0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4\": rpc error: code = NotFound desc = could not find container \"0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4\": container with ID starting with 0ddd6dde98c3056c8ac16ed440f05fa20d7b142e7f1cabd73ae14fcc5d787ed4 not found: ID does not exist"
Jul 07 13:07:31.693300 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.693278    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"]
Jul 07 13:07:31.697102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:31.697081    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-abdf58a0-6bcd58c6b6-6zhbj"]
Jul 07 13:07:32.517797 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:32.517711    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d0641e4-fba6-48df-9ef2-3b9685cdedc2" path="/var/lib/kubelet/pods/5d0641e4-fba6-48df-9ef2-3b9685cdedc2/volumes"
Jul 07 13:07:37.383073 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:37.383039    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:07:38.168382 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.168353    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"]
Jul 07 13:07:38.168689 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.168675    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d0641e4-fba6-48df-9ef2-3b9685cdedc2" containerName="maas-api"
Jul 07 13:07:38.168813 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.168691    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0641e4-fba6-48df-9ef2-3b9685cdedc2" containerName="maas-api"
Jul 07 13:07:38.168813 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.168755    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d0641e4-fba6-48df-9ef2-3b9685cdedc2" containerName="maas-api"
Jul 07 13:07:38.172924 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.172907    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.180229 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.180204    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a290d93f-serving-cert\""
Jul 07 13:07:38.181562 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.181547    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:07:38.183946 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.183931    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:07:38.190353 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.190338    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:07:38.196018 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.195992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"]
Jul 07 13:07:38.296422 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.296389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-openshift-service-ca\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.296622 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.296428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-794xl\" (UniqueName: \"kubernetes.io/projected/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-kube-api-access-794xl\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.296622 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.296530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-maas-api-tls\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.397772 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.397726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-openshift-service-ca\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.397772 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.397779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-794xl\" (UniqueName: \"kubernetes.io/projected/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-kube-api-access-794xl\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.398213 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.397843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-maas-api-tls\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.398480 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.398460    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-openshift-service-ca\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.400157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.400140    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-maas-api-tls\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.412663 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.412638    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-794xl\" (UniqueName: \"kubernetes.io/projected/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-kube-api-access-794xl\") pod \"maas-api-e2e-ait-a290d93f-575b586945-pc9x7\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.482653 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.482566    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:38.615790 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.615765    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"]
Jul 07 13:07:38.617581 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:07:38.617557    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc400498_d0cb_4032_ae6c_75e3c43e1dd8.slice/crio-4e9804b76dc49b80119212f64676c40af8c183260cd67f24be293843e4467524 WatchSource:0}: Error finding container 4e9804b76dc49b80119212f64676c40af8c183260cd67f24be293843e4467524: Status 404 returned error can't find the container with id 4e9804b76dc49b80119212f64676c40af8c183260cd67f24be293843e4467524
Jul 07 13:07:38.705914 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:38.705876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7" event={"ID":"cc400498-d0cb-4032-ae6c-75e3c43e1dd8","Type":"ContainerStarted","Data":"4e9804b76dc49b80119212f64676c40af8c183260cd67f24be293843e4467524"}
Jul 07 13:07:39.713140 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:39.713110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7" event={"ID":"cc400498-d0cb-4032-ae6c-75e3c43e1dd8","Type":"ContainerStarted","Data":"1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c"}
Jul 07 13:07:39.713566 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:39.713200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:39.731933 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:39.731883    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7" podStartSLOduration=1.348436201 podStartE2EDuration="1.731867579s" totalImagesPullingTime="383.431378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:07:39.729329588 +0000 UTC m=+1527.730196512" watchObservedRunningTime="2026-07-07 13:07:39.731867579 +0000 UTC m=+1527.732734499"
Jul 07 13:07:43.473339 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.473306    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"]
Jul 07 13:07:43.473831 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.473509    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7" podUID="cc400498-d0cb-4032-ae6c-75e3c43e1dd8" containerName="maas-api" containerID="cri-o://1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c" gracePeriod=30
Jul 07 13:07:43.714573 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.714548    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:43.733209 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.733134    2579 generic.go:356] "Generic (PLEG): container finished" podID="cc400498-d0cb-4032-ae6c-75e3c43e1dd8" containerID="1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c" exitCode=0
Jul 07 13:07:43.733354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.733205    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"
Jul 07 13:07:43.733354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.733226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7" event={"ID":"cc400498-d0cb-4032-ae6c-75e3c43e1dd8","Type":"ContainerDied","Data":"1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c"}
Jul 07 13:07:43.733354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.733291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7" event={"ID":"cc400498-d0cb-4032-ae6c-75e3c43e1dd8","Type":"ContainerDied","Data":"4e9804b76dc49b80119212f64676c40af8c183260cd67f24be293843e4467524"}
Jul 07 13:07:43.733354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.733322    2579 scope.go:117] "RemoveContainer" containerID="1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c"
Jul 07 13:07:43.742287 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.742267    2579 scope.go:117] "RemoveContainer" containerID="1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c"
Jul 07 13:07:43.742580 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:07:43.742562    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c\": container with ID starting with 1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c not found: ID does not exist" containerID="1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c"
Jul 07 13:07:43.742639 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.742589    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c"} err="failed to get container status \"1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c\": rpc error: code = NotFound desc = could not find container \"1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c\": container with ID starting with 1774138ec054db80867da7af9026eb1724600ba4d10f88072ff13121b17a431c not found: ID does not exist"
Jul 07 13:07:43.841620 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.841593    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-openshift-service-ca\") pod \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") "
Jul 07 13:07:43.841780 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.841644    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-794xl\" (UniqueName: \"kubernetes.io/projected/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-kube-api-access-794xl\") pod \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") "
Jul 07 13:07:43.841780 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.841669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-maas-api-tls\") pod \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\" (UID: \"cc400498-d0cb-4032-ae6c-75e3c43e1dd8\") "
Jul 07 13:07:43.841984 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.841961    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc400498-d0cb-4032-ae6c-75e3c43e1dd8" (UID: "cc400498-d0cb-4032-ae6c-75e3c43e1dd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:07:43.843677 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.843654    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-kube-api-access-794xl" (OuterVolumeSpecName: "kube-api-access-794xl") pod "cc400498-d0cb-4032-ae6c-75e3c43e1dd8" (UID: "cc400498-d0cb-4032-ae6c-75e3c43e1dd8"). InnerVolumeSpecName "kube-api-access-794xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:07:43.843779 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.843715    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc400498-d0cb-4032-ae6c-75e3c43e1dd8" (UID: "cc400498-d0cb-4032-ae6c-75e3c43e1dd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:07:43.942874 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.942835    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:07:43.942874 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.942864    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-794xl\" (UniqueName: \"kubernetes.io/projected/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-kube-api-access-794xl\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:07:43.942874 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:43.942879    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc400498-d0cb-4032-ae6c-75e3c43e1dd8-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:07:44.053421 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:44.053391    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"]
Jul 07 13:07:44.057045 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:44.057020    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a290d93f-575b586945-pc9x7"]
Jul 07 13:07:44.517721 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:44.517687    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc400498-d0cb-4032-ae6c-75e3c43e1dd8" path="/var/lib/kubelet/pods/cc400498-d0cb-4032-ae6c-75e3c43e1dd8/volumes"
Jul 07 13:07:57.138940 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.138900    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"]
Jul 07 13:07:57.139395 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.139259    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc400498-d0cb-4032-ae6c-75e3c43e1dd8" containerName="maas-api"
Jul 07 13:07:57.139395 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.139272    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc400498-d0cb-4032-ae6c-75e3c43e1dd8" containerName="maas-api"
Jul 07 13:07:57.139395 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.139321    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc400498-d0cb-4032-ae6c-75e3c43e1dd8" containerName="maas-api"
Jul 07 13:07:57.143510 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.143491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.146121 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.146092    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:07:57.146531 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.146496    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:07:57.146638 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.146551    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:07:57.146638 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.146508    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-087366f3-serving-cert\""
Jul 07 13:07:57.157354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.157329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"]
Jul 07 13:07:57.258080 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.258048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f5gq\" (UniqueName: \"kubernetes.io/projected/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-kube-api-access-7f5gq\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.258262 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.258085    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-maas-api-tls\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.258262 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.258146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-openshift-service-ca\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.358722 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.358688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-maas-api-tls\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.358895 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.358751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-openshift-service-ca\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.358895 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.358801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f5gq\" (UniqueName: \"kubernetes.io/projected/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-kube-api-access-7f5gq\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.359427 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.359400    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-openshift-service-ca\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.361227 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.361193    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-maas-api-tls\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.383788 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.383753    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f5gq\" (UniqueName: \"kubernetes.io/projected/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-kube-api-access-7f5gq\") pod \"maas-api-e2e-derive-087366f3-5b878f755f-jc68b\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.454295 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.454210    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:57.591750 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.591722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"]
Jul 07 13:07:57.593108 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:07:57.593082    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e13bdc_f17d_4ce6_a571_7d0f512b6212.slice/crio-aa6d8e598657cf3103f801ca43b18402cfb8554dd9b016372623911b1973973c WatchSource:0}: Error finding container aa6d8e598657cf3103f801ca43b18402cfb8554dd9b016372623911b1973973c: Status 404 returned error can't find the container with id aa6d8e598657cf3103f801ca43b18402cfb8554dd9b016372623911b1973973c
Jul 07 13:07:57.803394 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:57.803304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b" event={"ID":"e9e13bdc-f17d-4ce6-a571-7d0f512b6212","Type":"ContainerStarted","Data":"aa6d8e598657cf3103f801ca43b18402cfb8554dd9b016372623911b1973973c"}
Jul 07 13:07:58.809607 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:58.809570    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b" event={"ID":"e9e13bdc-f17d-4ce6-a571-7d0f512b6212","Type":"ContainerStarted","Data":"583c656936309cceeee24835b7906d46797fd8f5c690ba68dab0609a6673f0ae"}
Jul 07 13:07:58.809976 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:58.809757    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:07:58.828209 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:07:58.828158    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b" podStartSLOduration=1.4815996120000001 podStartE2EDuration="1.828142615s" totalImagesPullingTime="346.543003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:07:58.827130598 +0000 UTC m=+1546.827997517" watchObservedRunningTime="2026-07-07 13:07:58.828142615 +0000 UTC m=+1546.829009533"
Jul 07 13:08:01.705785 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:01.705754    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"]
Jul 07 13:08:01.706254 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:01.705997    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b" podUID="e9e13bdc-f17d-4ce6-a571-7d0f512b6212" containerName="maas-api" containerID="cri-o://583c656936309cceeee24835b7906d46797fd8f5c690ba68dab0609a6673f0ae" gracePeriod=30
Jul 07 13:08:01.825876 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:01.825843    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9e13bdc-f17d-4ce6-a571-7d0f512b6212" containerID="583c656936309cceeee24835b7906d46797fd8f5c690ba68dab0609a6673f0ae" exitCode=0
Jul 07 13:08:01.825986 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:01.825918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b" event={"ID":"e9e13bdc-f17d-4ce6-a571-7d0f512b6212","Type":"ContainerDied","Data":"583c656936309cceeee24835b7906d46797fd8f5c690ba68dab0609a6673f0ae"}
Jul 07 13:08:01.945338 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:01.945312    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:08:02.065561 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.065514    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:08:02.099678 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.099648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f5gq\" (UniqueName: \"kubernetes.io/projected/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-kube-api-access-7f5gq\") pod \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") "
Jul 07 13:08:02.099819 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.099690    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-openshift-service-ca\") pod \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") "
Jul 07 13:08:02.099819 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.099734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-maas-api-tls\") pod \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\" (UID: \"e9e13bdc-f17d-4ce6-a571-7d0f512b6212\") "
Jul 07 13:08:02.100138 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.100112    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9e13bdc-f17d-4ce6-a571-7d0f512b6212" (UID: "e9e13bdc-f17d-4ce6-a571-7d0f512b6212"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:08:02.101712 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.101690    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-kube-api-access-7f5gq" (OuterVolumeSpecName: "kube-api-access-7f5gq") pod "e9e13bdc-f17d-4ce6-a571-7d0f512b6212" (UID: "e9e13bdc-f17d-4ce6-a571-7d0f512b6212"). InnerVolumeSpecName "kube-api-access-7f5gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:08:02.101804 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.101721    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9e13bdc-f17d-4ce6-a571-7d0f512b6212" (UID: "e9e13bdc-f17d-4ce6-a571-7d0f512b6212"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:08:02.201140 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.201106    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f5gq\" (UniqueName: \"kubernetes.io/projected/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-kube-api-access-7f5gq\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:08:02.201140 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.201132    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:08:02.201140 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.201142    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e13bdc-f17d-4ce6-a571-7d0f512b6212-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:08:02.831877 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.831848    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"
Jul 07 13:08:02.832283 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.831853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b" event={"ID":"e9e13bdc-f17d-4ce6-a571-7d0f512b6212","Type":"ContainerDied","Data":"aa6d8e598657cf3103f801ca43b18402cfb8554dd9b016372623911b1973973c"}
Jul 07 13:08:02.832283 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.831980    2579 scope.go:117] "RemoveContainer" containerID="583c656936309cceeee24835b7906d46797fd8f5c690ba68dab0609a6673f0ae"
Jul 07 13:08:02.847292 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.847265    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"]
Jul 07 13:08:02.851302 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:02.851280    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-087366f3-5b878f755f-jc68b"]
Jul 07 13:08:04.517888 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:04.517849    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e13bdc-f17d-4ce6-a571-7d0f512b6212" path="/var/lib/kubelet/pods/e9e13bdc-f17d-4ce6-a571-7d0f512b6212/volumes"
Jul 07 13:08:08.441632 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:08:08.441597    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:09:30.872025 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:30.871988    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:09:38.680317 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:38.676024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:09:46.177353 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:46.177317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:09:48.685409 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.685375    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"]
Jul 07 13:09:48.685894 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.685807    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e13bdc-f17d-4ce6-a571-7d0f512b6212" containerName="maas-api"
Jul 07 13:09:48.685894 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.685823    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e13bdc-f17d-4ce6-a571-7d0f512b6212" containerName="maas-api"
Jul 07 13:09:48.686002 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.685913    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e13bdc-f17d-4ce6-a571-7d0f512b6212" containerName="maas-api"
Jul 07 13:09:48.688015 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.687996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.693390 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.693371    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3c66f600-serving-cert\""
Jul 07 13:09:48.693909 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.693863    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:09:48.694014 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.693900    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:09:48.694171 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.694148    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:09:48.704084 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.704051    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"]
Jul 07 13:09:48.777624 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.777596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c635294-80b9-4eb1-8a32-550b1117e59e-maas-api-tls\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.777624 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.777635    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b24qd\" (UniqueName: \"kubernetes.io/projected/8c635294-80b9-4eb1-8a32-550b1117e59e-kube-api-access-b24qd\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.777824 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.777662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c635294-80b9-4eb1-8a32-550b1117e59e-openshift-service-ca\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.878405 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.878373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24qd\" (UniqueName: \"kubernetes.io/projected/8c635294-80b9-4eb1-8a32-550b1117e59e-kube-api-access-b24qd\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.878576 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.878417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c635294-80b9-4eb1-8a32-550b1117e59e-openshift-service-ca\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.878576 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.878469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c635294-80b9-4eb1-8a32-550b1117e59e-maas-api-tls\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.879200 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.879182    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c635294-80b9-4eb1-8a32-550b1117e59e-openshift-service-ca\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.880760 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.880744    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c635294-80b9-4eb1-8a32-550b1117e59e-maas-api-tls\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.887160 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.887134    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b24qd\" (UniqueName: \"kubernetes.io/projected/8c635294-80b9-4eb1-8a32-550b1117e59e-kube-api-access-b24qd\") pod \"maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:48.998947 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:48.998877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:49.136333 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:09:49.136308    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c635294_80b9_4eb1_8a32_550b1117e59e.slice/crio-3431fc7e06c076724670b1eb0035336d445b5a82aaaffea53b4ba4b5cc58a9c4 WatchSource:0}: Error finding container 3431fc7e06c076724670b1eb0035336d445b5a82aaaffea53b4ba4b5cc58a9c4: Status 404 returned error can't find the container with id 3431fc7e06c076724670b1eb0035336d445b5a82aaaffea53b4ba4b5cc58a9c4
Jul 07 13:09:49.138083 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:49.138051    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"]
Jul 07 13:09:49.327560 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:49.327508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv" event={"ID":"8c635294-80b9-4eb1-8a32-550b1117e59e","Type":"ContainerStarted","Data":"3431fc7e06c076724670b1eb0035336d445b5a82aaaffea53b4ba4b5cc58a9c4"}
Jul 07 13:09:50.333379 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:50.333340    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv" event={"ID":"8c635294-80b9-4eb1-8a32-550b1117e59e","Type":"ContainerStarted","Data":"c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6"}
Jul 07 13:09:50.333752 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:50.333433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:09:50.366074 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:50.366031    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv" podStartSLOduration=1.8465718519999998 podStartE2EDuration="2.36601602s" totalImagesPullingTime="519.444168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:09:50.363705838 +0000 UTC m=+1658.364572756" watchObservedRunningTime="2026-07-07 13:09:50.36601602 +0000 UTC m=+1658.366882939"
Jul 07 13:09:54.100149 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:54.100114    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:09:56.343165 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:09:56.343134    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:10:01.551034 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.550998    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"]
Jul 07 13:10:01.551478 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.551238    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv" podUID="8c635294-80b9-4eb1-8a32-550b1117e59e" containerName="maas-api" containerID="cri-o://c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6" gracePeriod=30
Jul 07 13:10:01.781811 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.781771    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:10:01.868382 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.868362    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c635294-80b9-4eb1-8a32-550b1117e59e-openshift-service-ca\") pod \"8c635294-80b9-4eb1-8a32-550b1117e59e\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") "
Jul 07 13:10:01.868499 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.868397    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c635294-80b9-4eb1-8a32-550b1117e59e-maas-api-tls\") pod \"8c635294-80b9-4eb1-8a32-550b1117e59e\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") "
Jul 07 13:10:01.868499 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.868429    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b24qd\" (UniqueName: \"kubernetes.io/projected/8c635294-80b9-4eb1-8a32-550b1117e59e-kube-api-access-b24qd\") pod \"8c635294-80b9-4eb1-8a32-550b1117e59e\" (UID: \"8c635294-80b9-4eb1-8a32-550b1117e59e\") "
Jul 07 13:10:01.868789 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.868761    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c635294-80b9-4eb1-8a32-550b1117e59e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c635294-80b9-4eb1-8a32-550b1117e59e" (UID: "8c635294-80b9-4eb1-8a32-550b1117e59e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:10:01.870413 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.870385    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c635294-80b9-4eb1-8a32-550b1117e59e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c635294-80b9-4eb1-8a32-550b1117e59e" (UID: "8c635294-80b9-4eb1-8a32-550b1117e59e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:10:01.870485 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.870426    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c635294-80b9-4eb1-8a32-550b1117e59e-kube-api-access-b24qd" (OuterVolumeSpecName: "kube-api-access-b24qd") pod "8c635294-80b9-4eb1-8a32-550b1117e59e" (UID: "8c635294-80b9-4eb1-8a32-550b1117e59e"). InnerVolumeSpecName "kube-api-access-b24qd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:10:01.969373 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.969348    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b24qd\" (UniqueName: \"kubernetes.io/projected/8c635294-80b9-4eb1-8a32-550b1117e59e-kube-api-access-b24qd\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:10:01.969373 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.969369    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c635294-80b9-4eb1-8a32-550b1117e59e-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:10:01.969494 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:01.969378    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c635294-80b9-4eb1-8a32-550b1117e59e-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:10:02.391666 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.391629    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c635294-80b9-4eb1-8a32-550b1117e59e" containerID="c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6" exitCode=0
Jul 07 13:10:02.391898 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.391680    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"
Jul 07 13:10:02.391898 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.391716    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv" event={"ID":"8c635294-80b9-4eb1-8a32-550b1117e59e","Type":"ContainerDied","Data":"c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6"}
Jul 07 13:10:02.391898 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.391774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv" event={"ID":"8c635294-80b9-4eb1-8a32-550b1117e59e","Type":"ContainerDied","Data":"3431fc7e06c076724670b1eb0035336d445b5a82aaaffea53b4ba4b5cc58a9c4"}
Jul 07 13:10:02.391898 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.391796    2579 scope.go:117] "RemoveContainer" containerID="c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6"
Jul 07 13:10:02.401381 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.401364    2579 scope.go:117] "RemoveContainer" containerID="c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6"
Jul 07 13:10:02.401660 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:10:02.401642    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6\": container with ID starting with c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6 not found: ID does not exist" containerID="c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6"
Jul 07 13:10:02.401718 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.401667    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6"} err="failed to get container status \"c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6\": rpc error: code = NotFound desc = could not find container \"c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6\": container with ID starting with c61207bb262dcdd5aea9d86c4dd0bf16d306df9377668d89712d1f97c3b712a6 not found: ID does not exist"
Jul 07 13:10:02.416202 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.416181    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"]
Jul 07 13:10:02.420178 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.420158    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c66f600-74fc7b94cb-xcdxv"]
Jul 07 13:10:02.517778 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:02.517754    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c635294-80b9-4eb1-8a32-550b1117e59e" path="/var/lib/kubelet/pods/8c635294-80b9-4eb1-8a32-550b1117e59e/volumes"
Jul 07 13:10:05.383984 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:05.383947    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:10:31.791287 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.791257    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"]
Jul 07 13:10:31.791666 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.791573    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c635294-80b9-4eb1-8a32-550b1117e59e" containerName="maas-api"
Jul 07 13:10:31.791666 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.791585    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c635294-80b9-4eb1-8a32-550b1117e59e" containerName="maas-api"
Jul 07 13:10:31.791666 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.791642    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c635294-80b9-4eb1-8a32-550b1117e59e" containerName="maas-api"
Jul 07 13:10:31.793321 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.793305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.796870 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.796851    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:10:31.796997 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.796851    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7dc7bf8d-serving-cert\""
Jul 07 13:10:31.796997 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.796924    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:10:31.797448 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.797433    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:10:31.805563 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.805543    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"]
Jul 07 13:10:31.896474 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.896446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c59c124-c1da-4c44-a275-4dce9bda3169-openshift-service-ca\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.896594 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.896483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c59c124-c1da-4c44-a275-4dce9bda3169-maas-api-tls\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.896594 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.896511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-224s5\" (UniqueName: \"kubernetes.io/projected/4c59c124-c1da-4c44-a275-4dce9bda3169-kube-api-access-224s5\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.996893 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.996862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-224s5\" (UniqueName: \"kubernetes.io/projected/4c59c124-c1da-4c44-a275-4dce9bda3169-kube-api-access-224s5\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.997031 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.996946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c59c124-c1da-4c44-a275-4dce9bda3169-openshift-service-ca\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.997031 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.996981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c59c124-c1da-4c44-a275-4dce9bda3169-maas-api-tls\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.997596 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.997577    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c59c124-c1da-4c44-a275-4dce9bda3169-openshift-service-ca\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:31.999247 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:31.999226    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c59c124-c1da-4c44-a275-4dce9bda3169-maas-api-tls\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:32.012529 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:32.012489    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-224s5\" (UniqueName: \"kubernetes.io/projected/4c59c124-c1da-4c44-a275-4dce9bda3169-kube-api-access-224s5\") pod \"maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:32.103495 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:32.103465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:32.238470 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:32.238438    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"]
Jul 07 13:10:32.240829 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:10:32.240796    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c59c124_c1da_4c44_a275_4dce9bda3169.slice/crio-e2455cdaa45954e76c3705939fd4de32b903ba8e162acd60891024b30f77d74f WatchSource:0}: Error finding container e2455cdaa45954e76c3705939fd4de32b903ba8e162acd60891024b30f77d74f: Status 404 returned error can't find the container with id e2455cdaa45954e76c3705939fd4de32b903ba8e162acd60891024b30f77d74f
Jul 07 13:10:32.539484 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:32.539399    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6" event={"ID":"4c59c124-c1da-4c44-a275-4dce9bda3169","Type":"ContainerStarted","Data":"e2455cdaa45954e76c3705939fd4de32b903ba8e162acd60891024b30f77d74f"}
Jul 07 13:10:33.547731 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:33.547690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6" event={"ID":"4c59c124-c1da-4c44-a275-4dce9bda3169","Type":"ContainerStarted","Data":"9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141"}
Jul 07 13:10:33.548179 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:33.547827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:33.568949 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:33.568907    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6" podStartSLOduration=2.107139552 podStartE2EDuration="2.568893763s" totalImagesPullingTime="461.754211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:10:33.568197021 +0000 UTC m=+1701.569063940" watchObservedRunningTime="2026-07-07 13:10:33.568893763 +0000 UTC m=+1701.569760682"
Jul 07 13:10:38.272565 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:38.272476    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:10:39.556775 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:39.556747    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:40.857383 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.857350    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"]
Jul 07 13:10:40.859579 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.859561    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:40.861648 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.861622    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7dc7bf8d\"/\"openshift-service-ca.crt\""
Jul 07 13:10:40.862134 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.862115    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7dc7bf8d\"/\"e2e-lifecycle-model-7dc7bf8d-kserve-self-signed-certs\""
Jul 07 13:10:40.862239 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.862139    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7dc7bf8d\"/\"kube-root-ca.crt\""
Jul 07 13:10:40.862312 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.862293    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7dc7bf8d\"/\"default-dockercfg-v96dz\""
Jul 07 13:10:40.872241 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.872219    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"]
Jul 07 13:10:40.966897 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.966865    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-home\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:40.967048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.966897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-model-cache\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:40.967048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.966934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-dshm\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:40.967048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.966973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6dc8\" (UniqueName: \"kubernetes.io/projected/6aafdbf7-4662-48d3-9fa6-518b12b932a2-kube-api-access-n6dc8\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:40.967048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.967009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tls-certs\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:40.967048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:40.967037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tmp-dir\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.067747 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.067719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-home\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.067747 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.067750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-model-cache\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.067935 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.067777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-dshm\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.067935 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.067912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6dc8\" (UniqueName: \"kubernetes.io/projected/6aafdbf7-4662-48d3-9fa6-518b12b932a2-kube-api-access-n6dc8\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.068048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.067961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tls-certs\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.068048 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.068007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tmp-dir\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.068159 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.068091    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-model-cache\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.068159 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.068132    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-home\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.068352 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.068332    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tmp-dir\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.070047 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.070028    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-dshm\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.070413 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.070391    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tls-certs\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.076420 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.076394    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6dc8\" (UniqueName: \"kubernetes.io/projected/6aafdbf7-4662-48d3-9fa6-518b12b932a2-kube-api-access-n6dc8\") pod \"e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") " pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.168833 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.168769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:41.295300 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.295278    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"]
Jul 07 13:10:41.296942 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:10:41.296916    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aafdbf7_4662_48d3_9fa6_518b12b932a2.slice/crio-7eebf9c3d692e76bd03f71812d5f64d55c1a4f698d3ff71d7a89f3201f05cb53 WatchSource:0}: Error finding container 7eebf9c3d692e76bd03f71812d5f64d55c1a4f698d3ff71d7a89f3201f05cb53: Status 404 returned error can't find the container with id 7eebf9c3d692e76bd03f71812d5f64d55c1a4f698d3ff71d7a89f3201f05cb53
Jul 07 13:10:41.584015 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.583988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp" event={"ID":"6aafdbf7-4662-48d3-9fa6-518b12b932a2","Type":"ContainerStarted","Data":"dfd3fcd15cef8f117858c93357b36df59840cb48886b1190c8fd1cd62db3bd2c"}
Jul 07 13:10:41.584160 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.584023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp" event={"ID":"6aafdbf7-4662-48d3-9fa6-518b12b932a2","Type":"ContainerStarted","Data":"7eebf9c3d692e76bd03f71812d5f64d55c1a4f698d3ff71d7a89f3201f05cb53"}
Jul 07 13:10:41.605803 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:41.605749    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp" podStartSLOduration=1.605729748 podStartE2EDuration="1.605729748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:10:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:10:41.605147496 +0000 UTC m=+1709.606014415" watchObservedRunningTime="2026-07-07 13:10:41.605729748 +0000 UTC m=+1709.606596669"
Jul 07 13:10:49.879907 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:49.879873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:10:51.169739 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:51.169706    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:51.170133 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:51.169770    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:51.181896 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:51.181855    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:51.645184 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:51.645159    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:10:56.672689 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:56.672656    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:10:58.912607 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:58.912577    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"]
Jul 07 13:10:58.913066 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:58.912770    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6" podUID="4c59c124-c1da-4c44-a275-4dce9bda3169" containerName="maas-api" containerID="cri-o://9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141" gracePeriod=30
Jul 07 13:10:59.149093 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.149073    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:59.303605 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.303543    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c59c124-c1da-4c44-a275-4dce9bda3169-openshift-service-ca\") pod \"4c59c124-c1da-4c44-a275-4dce9bda3169\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") "
Jul 07 13:10:59.303605 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.303587    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c59c124-c1da-4c44-a275-4dce9bda3169-maas-api-tls\") pod \"4c59c124-c1da-4c44-a275-4dce9bda3169\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") "
Jul 07 13:10:59.303771 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.303615    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-224s5\" (UniqueName: \"kubernetes.io/projected/4c59c124-c1da-4c44-a275-4dce9bda3169-kube-api-access-224s5\") pod \"4c59c124-c1da-4c44-a275-4dce9bda3169\" (UID: \"4c59c124-c1da-4c44-a275-4dce9bda3169\") "
Jul 07 13:10:59.303877 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.303843    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c59c124-c1da-4c44-a275-4dce9bda3169-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c59c124-c1da-4c44-a275-4dce9bda3169" (UID: "4c59c124-c1da-4c44-a275-4dce9bda3169"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:10:59.305544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.305514    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c59c124-c1da-4c44-a275-4dce9bda3169-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c59c124-c1da-4c44-a275-4dce9bda3169" (UID: "4c59c124-c1da-4c44-a275-4dce9bda3169"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:10:59.305683 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.305670    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c59c124-c1da-4c44-a275-4dce9bda3169-kube-api-access-224s5" (OuterVolumeSpecName: "kube-api-access-224s5") pod "4c59c124-c1da-4c44-a275-4dce9bda3169" (UID: "4c59c124-c1da-4c44-a275-4dce9bda3169"). InnerVolumeSpecName "kube-api-access-224s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:10:59.404869 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.404840    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-224s5\" (UniqueName: \"kubernetes.io/projected/4c59c124-c1da-4c44-a275-4dce9bda3169-kube-api-access-224s5\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:10:59.404869 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.404864    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c59c124-c1da-4c44-a275-4dce9bda3169-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:10:59.405032 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.404878    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c59c124-c1da-4c44-a275-4dce9bda3169-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:10:59.671678 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.671651    2579 generic.go:356] "Generic (PLEG): container finished" podID="4c59c124-c1da-4c44-a275-4dce9bda3169" containerID="9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141" exitCode=0
Jul 07 13:10:59.671810 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.671691    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"
Jul 07 13:10:59.671810 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.671680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6" event={"ID":"4c59c124-c1da-4c44-a275-4dce9bda3169","Type":"ContainerDied","Data":"9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141"}
Jul 07 13:10:59.671810 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.671739    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6" event={"ID":"4c59c124-c1da-4c44-a275-4dce9bda3169","Type":"ContainerDied","Data":"e2455cdaa45954e76c3705939fd4de32b903ba8e162acd60891024b30f77d74f"}
Jul 07 13:10:59.671810 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.671760    2579 scope.go:117] "RemoveContainer" containerID="9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141"
Jul 07 13:10:59.680068 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.680053    2579 scope.go:117] "RemoveContainer" containerID="9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141"
Jul 07 13:10:59.680288 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:10:59.680270    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141\": container with ID starting with 9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141 not found: ID does not exist" containerID="9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141"
Jul 07 13:10:59.680334 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.680297    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141"} err="failed to get container status \"9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141\": rpc error: code = NotFound desc = could not find container \"9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141\": container with ID starting with 9ba4904cb381aaeaf421f71389b816bcd9bdf50f01597303486dbde87bc81141 not found: ID does not exist"
Jul 07 13:10:59.692770 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.692748    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"]
Jul 07 13:10:59.696563 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:10:59.696544    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dc7bf8d-559bfdd9c-n8fw6"]
Jul 07 13:11:00.517640 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:00.517612    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c59c124-c1da-4c44-a275-4dce9bda3169" path="/var/lib/kubelet/pods/4c59c124-c1da-4c44-a275-4dce9bda3169/volumes"
Jul 07 13:11:09.643170 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:09.643134    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"]
Jul 07 13:11:09.680984 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:09.680957    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:11:24.663459 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.663402    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp" podUID="6aafdbf7-4662-48d3-9fa6-518b12b932a2" containerName="main" containerID="cri-o://dfd3fcd15cef8f117858c93357b36df59840cb48886b1190c8fd1cd62db3bd2c" gracePeriod=45
Jul 07 13:11:24.784545 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.784504    2579 generic.go:356] "Generic (PLEG): container finished" podID="6aafdbf7-4662-48d3-9fa6-518b12b932a2" containerID="dfd3fcd15cef8f117858c93357b36df59840cb48886b1190c8fd1cd62db3bd2c" exitCode=0
Jul 07 13:11:24.784820 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.784553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp" event={"ID":"6aafdbf7-4662-48d3-9fa6-518b12b932a2","Type":"ContainerDied","Data":"dfd3fcd15cef8f117858c93357b36df59840cb48886b1190c8fd1cd62db3bd2c"}
Jul 07 13:11:24.908779 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.908758    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:11:24.986374 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986319    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tls-certs\") pod \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") "
Jul 07 13:11:24.986374 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6dc8\" (UniqueName: \"kubernetes.io/projected/6aafdbf7-4662-48d3-9fa6-518b12b932a2-kube-api-access-n6dc8\") pod \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") "
Jul 07 13:11:24.986547 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986387    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-home\") pod \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") "
Jul 07 13:11:24.986547 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986493    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-model-cache\") pod \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") "
Jul 07 13:11:24.986662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986592    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-dshm\") pod \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") "
Jul 07 13:11:24.986662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986634    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tmp-dir\") pod \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\" (UID: \"6aafdbf7-4662-48d3-9fa6-518b12b932a2\") "
Jul 07 13:11:24.986662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986641    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-home" (OuterVolumeSpecName: "home") pod "6aafdbf7-4662-48d3-9fa6-518b12b932a2" (UID: "6aafdbf7-4662-48d3-9fa6-518b12b932a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:11:24.986815 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986763    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "6aafdbf7-4662-48d3-9fa6-518b12b932a2" (UID: "6aafdbf7-4662-48d3-9fa6-518b12b932a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:11:24.986894 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986857    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:11:24.986977 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986894    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:11:24.986977 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.986906    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6aafdbf7-4662-48d3-9fa6-518b12b932a2" (UID: "6aafdbf7-4662-48d3-9fa6-518b12b932a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:11:24.988529 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.988495    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-dshm" (OuterVolumeSpecName: "dshm") pod "6aafdbf7-4662-48d3-9fa6-518b12b932a2" (UID: "6aafdbf7-4662-48d3-9fa6-518b12b932a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:11:24.988634 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.988509    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aafdbf7-4662-48d3-9fa6-518b12b932a2-kube-api-access-n6dc8" (OuterVolumeSpecName: "kube-api-access-n6dc8") pod "6aafdbf7-4662-48d3-9fa6-518b12b932a2" (UID: "6aafdbf7-4662-48d3-9fa6-518b12b932a2"). InnerVolumeSpecName "kube-api-access-n6dc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:11:24.988634 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:24.988497    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6aafdbf7-4662-48d3-9fa6-518b12b932a2" (UID: "6aafdbf7-4662-48d3-9fa6-518b12b932a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:11:25.087210 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.087190    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:11:25.087210 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.087208    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:11:25.087354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.087218    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aafdbf7-4662-48d3-9fa6-518b12b932a2-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:11:25.087354 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.087226    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6dc8\" (UniqueName: \"kubernetes.io/projected/6aafdbf7-4662-48d3-9fa6-518b12b932a2-kube-api-access-n6dc8\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:11:25.790995 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.790965    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"
Jul 07 13:11:25.790995 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.790977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp" event={"ID":"6aafdbf7-4662-48d3-9fa6-518b12b932a2","Type":"ContainerDied","Data":"7eebf9c3d692e76bd03f71812d5f64d55c1a4f698d3ff71d7a89f3201f05cb53"}
Jul 07 13:11:25.791538 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.791022    2579 scope.go:117] "RemoveContainer" containerID="dfd3fcd15cef8f117858c93357b36df59840cb48886b1190c8fd1cd62db3bd2c"
Jul 07 13:11:25.814970 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.814947    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"]
Jul 07 13:11:25.818886 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:25.818867    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7dc7bf8d/e2e-lifecycle-model-7dc7bf8d-kserve-67c7649c5f-zw4tp"]
Jul 07 13:11:26.517460 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:26.517427    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aafdbf7-4662-48d3-9fa6-518b12b932a2" path="/var/lib/kubelet/pods/6aafdbf7-4662-48d3-9fa6-518b12b932a2/volumes"
Jul 07 13:11:48.996164 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:11:48.996130    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:12:04.577243 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:12:04.577160    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:12:16.113145 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:12:16.113104    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:13:01.669182 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669150    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"]
Jul 07 13:13:01.669624 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669615    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aafdbf7-4662-48d3-9fa6-518b12b932a2" containerName="main"
Jul 07 13:13:01.669679 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669630    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aafdbf7-4662-48d3-9fa6-518b12b932a2" containerName="main"
Jul 07 13:13:01.669679 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669659    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c59c124-c1da-4c44-a275-4dce9bda3169" containerName="maas-api"
Jul 07 13:13:01.669679 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669667    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c59c124-c1da-4c44-a275-4dce9bda3169" containerName="maas-api"
Jul 07 13:13:01.669788 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669736    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c59c124-c1da-4c44-a275-4dce9bda3169" containerName="maas-api"
Jul 07 13:13:01.669788 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.669747    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aafdbf7-4662-48d3-9fa6-518b12b932a2" containerName="main"
Jul 07 13:13:01.672849 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.672832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.674914 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.674892    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6b8022-serving-cert\""
Jul 07 13:13:01.674914 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.674905    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:13:01.675098 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.674987    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:13:01.675725 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.675705    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:13:01.685015 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.684995    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"]
Jul 07 13:13:01.703288 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.703261    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.703400 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.703310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.703400 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.703343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm8gj\" (UniqueName: \"kubernetes.io/projected/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-kube-api-access-xm8gj\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.803960 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.803931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.804122 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.803970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.804122 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.803999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm8gj\" (UniqueName: \"kubernetes.io/projected/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-kube-api-access-xm8gj\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.804122 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:13:01.804109    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-serving-cert: secret "maas-api-e2e-inf-a-6b8022-serving-cert" not found
Jul 07 13:13:01.804250 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:13:01.804194    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls podName:22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4 nodeName:}" failed. No retries permitted until 2026-07-07 13:13:02.304173387 +0000 UTC m=+1850.305040289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls") pod "maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" (UID: "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4") : secret "maas-api-e2e-inf-a-6b8022-serving-cert" not found
Jul 07 13:13:01.804577 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.804560    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:01.816715 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:01.816690    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm8gj\" (UniqueName: \"kubernetes.io/projected/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-kube-api-access-xm8gj\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:02.309151 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:02.309118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:02.311335 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:02.311304    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls\") pod \"maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:02.586021 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:02.585994    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:02.714925 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:02.714901    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"]
Jul 07 13:13:02.716831 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:13:02.716802    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22836f9f_0ac0_4abd_8ceb_f8feaa12f0c4.slice/crio-828f0bf0aa24c3b6a55eeee92ca44d80fc16c37cdfeaa094f55d90f34d26d631 WatchSource:0}: Error finding container 828f0bf0aa24c3b6a55eeee92ca44d80fc16c37cdfeaa094f55d90f34d26d631: Status 404 returned error can't find the container with id 828f0bf0aa24c3b6a55eeee92ca44d80fc16c37cdfeaa094f55d90f34d26d631
Jul 07 13:13:02.718724 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:02.718706    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:13:03.247426 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:03.247396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" event={"ID":"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4","Type":"ContainerStarted","Data":"828f0bf0aa24c3b6a55eeee92ca44d80fc16c37cdfeaa094f55d90f34d26d631"}
Jul 07 13:13:04.253218 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:04.253185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" event={"ID":"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4","Type":"ContainerStarted","Data":"a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9"}
Jul 07 13:13:04.253609 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:04.253301    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:04.283697 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:04.283654    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" podStartSLOduration=2.785590967 podStartE2EDuration="3.283642049s" totalImagesPullingTime="498.051082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:04.281059885 +0000 UTC m=+1852.281926803" watchObservedRunningTime="2026-07-07 13:13:04.283642049 +0000 UTC m=+1852.284508968"
Jul 07 13:13:08.974721 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:08.974686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:13:09.505327 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.505295    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"]
Jul 07 13:13:09.508714 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.508691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.510780 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.510755    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e2f0d5-serving-cert\""
Jul 07 13:13:09.521365 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.521344    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"]
Jul 07 13:13:09.567923 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.567902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7faa81ff-5e22-4eb3-b312-0db533ca356a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.568024 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.567934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7faa81ff-5e22-4eb3-b312-0db533ca356a-maas-api-tls\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.568072 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.568029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7z7b\" (UniqueName: \"kubernetes.io/projected/7faa81ff-5e22-4eb3-b312-0db533ca356a-kube-api-access-r7z7b\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.669107 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.669080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7faa81ff-5e22-4eb3-b312-0db533ca356a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.669265 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.669113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7faa81ff-5e22-4eb3-b312-0db533ca356a-maas-api-tls\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.669265 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.669164    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7z7b\" (UniqueName: \"kubernetes.io/projected/7faa81ff-5e22-4eb3-b312-0db533ca356a-kube-api-access-r7z7b\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.669737 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.669718    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7faa81ff-5e22-4eb3-b312-0db533ca356a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.671680 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.671654    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7faa81ff-5e22-4eb3-b312-0db533ca356a-maas-api-tls\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.679481 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.679452    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7z7b\" (UniqueName: \"kubernetes.io/projected/7faa81ff-5e22-4eb3-b312-0db533ca356a-kube-api-access-r7z7b\") pod \"maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.819605 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.819577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:09.950129 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:09.950106    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"]
Jul 07 13:13:09.952383 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:13:09.952352    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7faa81ff_5e22_4eb3_b312_0db533ca356a.slice/crio-ad688c534d2120665a565f7b2ea4baeaff2bc467822e4b3aadc938fcd45aa48f WatchSource:0}: Error finding container ad688c534d2120665a565f7b2ea4baeaff2bc467822e4b3aadc938fcd45aa48f: Status 404 returned error can't find the container with id ad688c534d2120665a565f7b2ea4baeaff2bc467822e4b3aadc938fcd45aa48f
Jul 07 13:13:10.263513 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:10.263433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:13:10.282557 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:10.282531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7" event={"ID":"7faa81ff-5e22-4eb3-b312-0db533ca356a","Type":"ContainerStarted","Data":"ad688c534d2120665a565f7b2ea4baeaff2bc467822e4b3aadc938fcd45aa48f"}
Jul 07 13:13:11.289350 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:11.289302    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7" event={"ID":"7faa81ff-5e22-4eb3-b312-0db533ca356a","Type":"ContainerStarted","Data":"590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f"}
Jul 07 13:13:11.289749 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:11.289417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:11.309150 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:11.309109    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7" podStartSLOduration=1.894984981 podStartE2EDuration="2.309095663s" totalImagesPullingTime="414.110682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:11.307299166 +0000 UTC m=+1859.308166085" watchObservedRunningTime="2026-07-07 13:13:11.309095663 +0000 UTC m=+1859.309962590"
Jul 07 13:13:17.299111 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:17.299080    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:13:19.267220 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:19.267185    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:13:32.674581 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.674548    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"]
Jul 07 13:13:32.678094 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.678076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.680226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.680202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6b8022\"/\"kube-root-ca.crt\""
Jul 07 13:13:32.680935 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.680911    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6b8022\"/\"test-model-6b8022-kserve-self-signed-certs\""
Jul 07 13:13:32.681044 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.680965    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6b8022\"/\"openshift-service-ca.crt\""
Jul 07 13:13:32.681044 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.680975    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6b8022\"/\"default-dockercfg-zk6cw\""
Jul 07 13:13:32.689051 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.689029    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"]
Jul 07 13:13:32.861322 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.861285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-dshm\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.861322 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.861323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqt6b\" (UniqueName: \"kubernetes.io/projected/73825224-aef4-4d39-9787-a6ce186e9333-kube-api-access-nqt6b\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.861592 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.861357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73825224-aef4-4d39-9787-a6ce186e9333-tls-certs\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.861592 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.861419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-model-cache\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.861592 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.861451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-home\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.861592 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.861473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-tmp-dir\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962512 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-dshm\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962512 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962457    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqt6b\" (UniqueName: \"kubernetes.io/projected/73825224-aef4-4d39-9787-a6ce186e9333-kube-api-access-nqt6b\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962512 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73825224-aef4-4d39-9787-a6ce186e9333-tls-certs\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962767 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-model-cache\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962767 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-home\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962767 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-tmp-dir\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.962998 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.962973    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-model-cache\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.963073 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.963012    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-tmp-dir\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.963073 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.963038    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-home\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.964821 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.964799    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-dshm\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.964974 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.964956    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73825224-aef4-4d39-9787-a6ce186e9333-tls-certs\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.970861 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.970837    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqt6b\" (UniqueName: \"kubernetes.io/projected/73825224-aef4-4d39-9787-a6ce186e9333-kube-api-access-nqt6b\") pod \"test-model-6b8022-kserve-b5f55bd85-hbnkf\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") " pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:32.988881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:32.988858    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:33.125313 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:33.125284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"]
Jul 07 13:13:33.127907 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:13:33.127873    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73825224_aef4_4d39_9787_a6ce186e9333.slice/crio-5ef735393bdac61d786c6a9a530d976f1319ed0a333328c31d420c9781a1a3ee WatchSource:0}: Error finding container 5ef735393bdac61d786c6a9a530d976f1319ed0a333328c31d420c9781a1a3ee: Status 404 returned error can't find the container with id 5ef735393bdac61d786c6a9a530d976f1319ed0a333328c31d420c9781a1a3ee
Jul 07 13:13:33.396399 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:33.396365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf" event={"ID":"73825224-aef4-4d39-9787-a6ce186e9333","Type":"ContainerStarted","Data":"3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f"}
Jul 07 13:13:33.396595 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:33.396410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf" event={"ID":"73825224-aef4-4d39-9787-a6ce186e9333","Type":"ContainerStarted","Data":"5ef735393bdac61d786c6a9a530d976f1319ed0a333328c31d420c9781a1a3ee"}
Jul 07 13:13:33.414102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:33.414056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf" podStartSLOduration=1.414040746 podStartE2EDuration="1.414040746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:33.41301311 +0000 UTC m=+1881.413880028" watchObservedRunningTime="2026-07-07 13:13:33.414040746 +0000 UTC m=+1881.414907664"
Jul 07 13:13:36.369968 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:36.369890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:13:42.989008 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:42.988969    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:42.989546 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:42.989050    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:43.001440 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:43.001416    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:43.454288 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:43.454260    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:13:47.326329 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:13:47.326295    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:14:00.077996 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.077963    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"]
Jul 07 13:14:00.081099 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.081080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.083579 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.083552    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e2f0d5\"/\"openshift-service-ca.crt\""
Jul 07 13:14:00.083715 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.083607    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e2f0d5\"/\"kube-root-ca.crt\""
Jul 07 13:14:00.083715 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.083643    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e2f0d5\"/\"test-model-e2f0d5-kserve-self-signed-certs\""
Jul 07 13:14:00.083824 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.083724    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e2f0d5\"/\"default-dockercfg-rvxxm\""
Jul 07 13:14:00.091240 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.091217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"]
Jul 07 13:14:00.155218 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.155188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-home\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.155405 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.155226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-dshm\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.155405 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.155262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q45lz\" (UniqueName: \"kubernetes.io/projected/ea3df2f2-3b56-4384-af33-7df274289faa-kube-api-access-q45lz\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.155405 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.155311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3df2f2-3b56-4384-af33-7df274289faa-tls-certs\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.155405 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.155337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-tmp-dir\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.155405 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.155383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-model-cache\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.255989 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.255954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3df2f2-3b56-4384-af33-7df274289faa-tls-certs\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.255989 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.255989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-tmp-dir\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256166 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-model-cache\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256166 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-home\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256166 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-dshm\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256166 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q45lz\" (UniqueName: \"kubernetes.io/projected/ea3df2f2-3b56-4384-af33-7df274289faa-kube-api-access-q45lz\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256440 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-tmp-dir\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256488 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256451    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-home\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.256548 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.256499    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-model-cache\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.258303 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.258284    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-dshm\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.258610 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.258582    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3df2f2-3b56-4384-af33-7df274289faa-tls-certs\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.264627 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.264603    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q45lz\" (UniqueName: \"kubernetes.io/projected/ea3df2f2-3b56-4384-af33-7df274289faa-kube-api-access-q45lz\") pod \"test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") " pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.391697 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.391622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:00.522891 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:00.522867    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"]
Jul 07 13:14:00.525326 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:14:00.525300    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea3df2f2_3b56_4384_af33_7df274289faa.slice/crio-f4b5efa0d43afef59a0ca9701cfca57e0a77861d24f9599ffec2ae38ca5302c3 WatchSource:0}: Error finding container f4b5efa0d43afef59a0ca9701cfca57e0a77861d24f9599ffec2ae38ca5302c3: Status 404 returned error can't find the container with id f4b5efa0d43afef59a0ca9701cfca57e0a77861d24f9599ffec2ae38ca5302c3
Jul 07 13:14:01.526848 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:01.526802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb" event={"ID":"ea3df2f2-3b56-4384-af33-7df274289faa","Type":"ContainerStarted","Data":"f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55"}
Jul 07 13:14:01.526848 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:01.526843    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb" event={"ID":"ea3df2f2-3b56-4384-af33-7df274289faa","Type":"ContainerStarted","Data":"f4b5efa0d43afef59a0ca9701cfca57e0a77861d24f9599ffec2ae38ca5302c3"}
Jul 07 13:14:01.545672 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:01.545627    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb" podStartSLOduration=1.5456131069999999 podStartE2EDuration="1.545613107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:14:01.544803893 +0000 UTC m=+1909.545670812" watchObservedRunningTime="2026-07-07 13:14:01.545613107 +0000 UTC m=+1909.546480026"
Jul 07 13:14:04.974597 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:04.974548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:14:10.392590 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:10.392552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:10.392590 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:10.392603    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:10.404987 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:10.404964    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:10.583240 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:10.583212    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:17.725407 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:17.725372    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"]
Jul 07 13:14:18.738681 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:18.738652    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"]
Jul 07 13:14:19.854732 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:19.854699    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"]
Jul 07 13:14:19.855185 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:19.854939    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" podUID="22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" containerName="maas-api" containerID="cri-o://a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9" gracePeriod=30
Jul 07 13:14:20.098747 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.098725    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:14:20.212662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.212579    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls\") pod \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") "
Jul 07 13:14:20.212662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.212661    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm8gj\" (UniqueName: \"kubernetes.io/projected/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-kube-api-access-xm8gj\") pod \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") "
Jul 07 13:14:20.212889 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.212683    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-openshift-service-ca\") pod \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\" (UID: \"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4\") "
Jul 07 13:14:20.213097 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.213069    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" (UID: "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:14:20.214792 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.214760    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" (UID: "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:14:20.214892 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.214836    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-kube-api-access-xm8gj" (OuterVolumeSpecName: "kube-api-access-xm8gj") pod "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" (UID: "22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4"). InnerVolumeSpecName "kube-api-access-xm8gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:14:20.313241 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.313208    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:20.313241 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.313235    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm8gj\" (UniqueName: \"kubernetes.io/projected/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-kube-api-access-xm8gj\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:20.313241 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.313249    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:20.618910 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.618880    2579 generic.go:356] "Generic (PLEG): container finished" podID="22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" containerID="a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9" exitCode=0
Jul 07 13:14:20.619102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.618917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" event={"ID":"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4","Type":"ContainerDied","Data":"a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9"}
Jul 07 13:14:20.619102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.618962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz" event={"ID":"22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4","Type":"ContainerDied","Data":"828f0bf0aa24c3b6a55eeee92ca44d80fc16c37cdfeaa094f55d90f34d26d631"}
Jul 07 13:14:20.619102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.618989    2579 scope.go:117] "RemoveContainer" containerID="a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9"
Jul 07 13:14:20.619102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.618919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"
Jul 07 13:14:20.627636 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.627621    2579 scope.go:117] "RemoveContainer" containerID="a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9"
Jul 07 13:14:20.627890 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:14:20.627870    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9\": container with ID starting with a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9 not found: ID does not exist" containerID="a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9"
Jul 07 13:14:20.627985 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.627894    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9"} err="failed to get container status \"a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9\": rpc error: code = NotFound desc = could not find container \"a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9\": container with ID starting with a6d4d446639d8f1674ee863a1a60a93294557a727ce84fac3c89cdf096996cf9 not found: ID does not exist"
Jul 07 13:14:20.634106 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.634083    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"]
Jul 07 13:14:20.639002 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:20.638983    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6b8022-5f6665b94-r45tz"]
Jul 07 13:14:22.517441 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:22.517407    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" path="/var/lib/kubelet/pods/22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4/volumes"
Jul 07 13:14:23.074223 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:23.074191    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:14:32.744991 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:32.744928    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf" podUID="73825224-aef4-4d39-9787-a6ce186e9333" containerName="main" containerID="cri-o://3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f" gracePeriod=45
Jul 07 13:14:32.999847 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:32.999778    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:14:33.126648 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-tmp-dir\") pod \"73825224-aef4-4d39-9787-a6ce186e9333\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") "
Jul 07 13:14:33.126648 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126652    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-home\") pod \"73825224-aef4-4d39-9787-a6ce186e9333\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") "
Jul 07 13:14:33.126920 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126679    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-model-cache\") pod \"73825224-aef4-4d39-9787-a6ce186e9333\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") "
Jul 07 13:14:33.126920 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126730    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-dshm\") pod \"73825224-aef4-4d39-9787-a6ce186e9333\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") "
Jul 07 13:14:33.126920 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqt6b\" (UniqueName: \"kubernetes.io/projected/73825224-aef4-4d39-9787-a6ce186e9333-kube-api-access-nqt6b\") pod \"73825224-aef4-4d39-9787-a6ce186e9333\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") "
Jul 07 13:14:33.126920 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126846    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73825224-aef4-4d39-9787-a6ce186e9333-tls-certs\") pod \"73825224-aef4-4d39-9787-a6ce186e9333\" (UID: \"73825224-aef4-4d39-9787-a6ce186e9333\") "
Jul 07 13:14:33.127094 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126942    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73825224-aef4-4d39-9787-a6ce186e9333" (UID: "73825224-aef4-4d39-9787-a6ce186e9333"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:33.127094 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.126960    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-home" (OuterVolumeSpecName: "home") pod "73825224-aef4-4d39-9787-a6ce186e9333" (UID: "73825224-aef4-4d39-9787-a6ce186e9333"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:33.127094 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.127019    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-model-cache" (OuterVolumeSpecName: "model-cache") pod "73825224-aef4-4d39-9787-a6ce186e9333" (UID: "73825224-aef4-4d39-9787-a6ce186e9333"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:33.127222 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.127169    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:33.127222 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.127188    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:33.127222 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.127197    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:33.129128 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.129095    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-dshm" (OuterVolumeSpecName: "dshm") pod "73825224-aef4-4d39-9787-a6ce186e9333" (UID: "73825224-aef4-4d39-9787-a6ce186e9333"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:33.129128 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.129119    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73825224-aef4-4d39-9787-a6ce186e9333-kube-api-access-nqt6b" (OuterVolumeSpecName: "kube-api-access-nqt6b") pod "73825224-aef4-4d39-9787-a6ce186e9333" (UID: "73825224-aef4-4d39-9787-a6ce186e9333"). InnerVolumeSpecName "kube-api-access-nqt6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:14:33.129316 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.129125    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73825224-aef4-4d39-9787-a6ce186e9333-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73825224-aef4-4d39-9787-a6ce186e9333" (UID: "73825224-aef4-4d39-9787-a6ce186e9333"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:14:33.227873 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.227834    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73825224-aef4-4d39-9787-a6ce186e9333-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:33.227873 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.227870    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqt6b\" (UniqueName: \"kubernetes.io/projected/73825224-aef4-4d39-9787-a6ce186e9333-kube-api-access-nqt6b\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:33.227873 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.227882    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73825224-aef4-4d39-9787-a6ce186e9333-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:33.684809 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.684774    2579 generic.go:356] "Generic (PLEG): container finished" podID="73825224-aef4-4d39-9787-a6ce186e9333" containerID="3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f" exitCode=0
Jul 07 13:14:33.684966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.684818    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"
Jul 07 13:14:33.684966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.684859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf" event={"ID":"73825224-aef4-4d39-9787-a6ce186e9333","Type":"ContainerDied","Data":"3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f"}
Jul 07 13:14:33.684966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.684905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf" event={"ID":"73825224-aef4-4d39-9787-a6ce186e9333","Type":"ContainerDied","Data":"5ef735393bdac61d786c6a9a530d976f1319ed0a333328c31d420c9781a1a3ee"}
Jul 07 13:14:33.684966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.684923    2579 scope.go:117] "RemoveContainer" containerID="3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f"
Jul 07 13:14:33.694009 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.693990    2579 scope.go:117] "RemoveContainer" containerID="3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f"
Jul 07 13:14:33.694315 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:14:33.694294    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f\": container with ID starting with 3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f not found: ID does not exist" containerID="3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f"
Jul 07 13:14:33.694415 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.694323    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f"} err="failed to get container status \"3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f\": rpc error: code = NotFound desc = could not find container \"3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f\": container with ID starting with 3500870e67cded6473b7348f7446147cc80902143bdc1d3d99bc3aead7607c4f not found: ID does not exist"
Jul 07 13:14:33.705482 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.705451    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"]
Jul 07 13:14:33.709406 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.709381    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6b8022/test-model-6b8022-kserve-b5f55bd85-hbnkf"]
Jul 07 13:14:33.758507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:33.758467    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb" podUID="ea3df2f2-3b56-4384-af33-7df274289faa" containerName="main" containerID="cri-o://f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55" gracePeriod=45
Jul 07 13:14:34.018157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.018130    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:34.134185 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134156    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-home\") pod \"ea3df2f2-3b56-4384-af33-7df274289faa\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") "
Jul 07 13:14:34.134384 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3df2f2-3b56-4384-af33-7df274289faa-tls-certs\") pod \"ea3df2f2-3b56-4384-af33-7df274289faa\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") "
Jul 07 13:14:34.134384 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134258    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-model-cache\") pod \"ea3df2f2-3b56-4384-af33-7df274289faa\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") "
Jul 07 13:14:34.134384 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134282    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-dshm\") pod \"ea3df2f2-3b56-4384-af33-7df274289faa\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") "
Jul 07 13:14:34.134384 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134299    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q45lz\" (UniqueName: \"kubernetes.io/projected/ea3df2f2-3b56-4384-af33-7df274289faa-kube-api-access-q45lz\") pod \"ea3df2f2-3b56-4384-af33-7df274289faa\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") "
Jul 07 13:14:34.134384 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134324    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-tmp-dir\") pod \"ea3df2f2-3b56-4384-af33-7df274289faa\" (UID: \"ea3df2f2-3b56-4384-af33-7df274289faa\") "
Jul 07 13:14:34.134689 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134452    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-home" (OuterVolumeSpecName: "home") pod "ea3df2f2-3b56-4384-af33-7df274289faa" (UID: "ea3df2f2-3b56-4384-af33-7df274289faa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:34.134689 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134568    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea3df2f2-3b56-4384-af33-7df274289faa" (UID: "ea3df2f2-3b56-4384-af33-7df274289faa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:34.134689 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.134595    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea3df2f2-3b56-4384-af33-7df274289faa" (UID: "ea3df2f2-3b56-4384-af33-7df274289faa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:34.136639 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.136614    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-dshm" (OuterVolumeSpecName: "dshm") pod "ea3df2f2-3b56-4384-af33-7df274289faa" (UID: "ea3df2f2-3b56-4384-af33-7df274289faa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:14:34.136776 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.136664    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea3df2f2-3b56-4384-af33-7df274289faa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea3df2f2-3b56-4384-af33-7df274289faa" (UID: "ea3df2f2-3b56-4384-af33-7df274289faa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:14:34.136776 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.136667    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3df2f2-3b56-4384-af33-7df274289faa-kube-api-access-q45lz" (OuterVolumeSpecName: "kube-api-access-q45lz") pod "ea3df2f2-3b56-4384-af33-7df274289faa" (UID: "ea3df2f2-3b56-4384-af33-7df274289faa"). InnerVolumeSpecName "kube-api-access-q45lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:14:34.235881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.235843    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:34.235881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.235876    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:34.235881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.235890    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3df2f2-3b56-4384-af33-7df274289faa-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:34.236125 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.235900    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:34.236125 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.235907    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3df2f2-3b56-4384-af33-7df274289faa-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:34.236125 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.235916    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q45lz\" (UniqueName: \"kubernetes.io/projected/ea3df2f2-3b56-4384-af33-7df274289faa-kube-api-access-q45lz\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:14:34.518083 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.518047    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73825224-aef4-4d39-9787-a6ce186e9333" path="/var/lib/kubelet/pods/73825224-aef4-4d39-9787-a6ce186e9333/volumes"
Jul 07 13:14:34.692488 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.692448    2579 generic.go:356] "Generic (PLEG): container finished" podID="ea3df2f2-3b56-4384-af33-7df274289faa" containerID="f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55" exitCode=0
Jul 07 13:14:34.692488 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.692498    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"
Jul 07 13:14:34.692786 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.692549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb" event={"ID":"ea3df2f2-3b56-4384-af33-7df274289faa","Type":"ContainerDied","Data":"f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55"}
Jul 07 13:14:34.692786 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.692607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb" event={"ID":"ea3df2f2-3b56-4384-af33-7df274289faa","Type":"ContainerDied","Data":"f4b5efa0d43afef59a0ca9701cfca57e0a77861d24f9599ffec2ae38ca5302c3"}
Jul 07 13:14:34.692786 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.692630    2579 scope.go:117] "RemoveContainer" containerID="f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55"
Jul 07 13:14:34.700952 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.700933    2579 scope.go:117] "RemoveContainer" containerID="f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55"
Jul 07 13:14:34.701261 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:14:34.701241    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55\": container with ID starting with f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55 not found: ID does not exist" containerID="f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55"
Jul 07 13:14:34.701303 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.701268    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55"} err="failed to get container status \"f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55\": rpc error: code = NotFound desc = could not find container \"f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55\": container with ID starting with f1f37ec9c331fe248d8f3d3b1dd0fbc687fb28028266c029dd170a1bf0725e55 not found: ID does not exist"
Jul 07 13:14:34.707497 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.707473    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"]
Jul 07 13:14:34.711012 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:34.710989    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e2f0d5/test-model-e2f0d5-kserve-5c4bdc6f9c-fcrgb"]
Jul 07 13:14:36.517496 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:14:36.517461    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea3df2f2-3b56-4384-af33-7df274289faa" path="/var/lib/kubelet/pods/ea3df2f2-3b56-4384-af33-7df274289faa/volumes"
Jul 07 13:15:00.148067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148030    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"]
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148368    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" containerName="maas-api"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148380    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" containerName="maas-api"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148390    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73825224-aef4-4d39-9787-a6ce186e9333" containerName="main"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148395    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="73825224-aef4-4d39-9787-a6ce186e9333" containerName="main"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148402    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3df2f2-3b56-4384-af33-7df274289faa" containerName="main"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148408    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3df2f2-3b56-4384-af33-7df274289faa" containerName="main"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148455    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="22836f9f-0ac0-4abd-8ceb-f8feaa12f0c4" containerName="maas-api"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148463    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="73825224-aef4-4d39-9787-a6ce186e9333" containerName="main"
Jul 07 13:15:00.148544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.148469    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3df2f2-3b56-4384-af33-7df274289faa" containerName="main"
Jul 07 13:15:00.151472 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.151455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:00.160194 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.160163    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"]
Jul 07 13:15:00.252306 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.252261    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn4j2\" (UniqueName: \"kubernetes.io/projected/9683d98c-40f5-419a-8b27-ea8ce4ec4756-kube-api-access-nn4j2\") pod \"maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t\" (UID: \"9683d98c-40f5-419a-8b27-ea8ce4ec4756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:00.353623 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.353584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn4j2\" (UniqueName: \"kubernetes.io/projected/9683d98c-40f5-419a-8b27-ea8ce4ec4756-kube-api-access-nn4j2\") pod \"maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t\" (UID: \"9683d98c-40f5-419a-8b27-ea8ce4ec4756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:00.360743 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.360711    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn4j2\" (UniqueName: \"kubernetes.io/projected/9683d98c-40f5-419a-8b27-ea8ce4ec4756-kube-api-access-nn4j2\") pod \"maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t\" (UID: \"9683d98c-40f5-419a-8b27-ea8ce4ec4756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:00.463921 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.463827    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:00.602924 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.602892    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"]
Jul 07 13:15:00.605217 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:15:00.605188    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9683d98c_40f5_419a_8b27_ea8ce4ec4756.slice/crio-9676d94dc6bbc7d31e60fbfe22baf7a600ebe77569949640238b605f4e738ef3 WatchSource:0}: Error finding container 9676d94dc6bbc7d31e60fbfe22baf7a600ebe77569949640238b605f4e738ef3: Status 404 returned error can't find the container with id 9676d94dc6bbc7d31e60fbfe22baf7a600ebe77569949640238b605f4e738ef3
Jul 07 13:15:00.823758 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:00.823721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t" event={"ID":"9683d98c-40f5-419a-8b27-ea8ce4ec4756","Type":"ContainerStarted","Data":"9676d94dc6bbc7d31e60fbfe22baf7a600ebe77569949640238b605f4e738ef3"}
Jul 07 13:15:03.007224 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.007188    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"]
Jul 07 13:15:03.007737 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.007441    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7" podUID="7faa81ff-5e22-4eb3-b312-0db533ca356a" containerName="maas-api" containerID="cri-o://590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f" gracePeriod=30
Jul 07 13:15:03.037585 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.037551    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"]
Jul 07 13:15:03.463510 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.463482    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:15:03.482441 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.482406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7faa81ff-5e22-4eb3-b312-0db533ca356a-openshift-service-ca\") pod \"7faa81ff-5e22-4eb3-b312-0db533ca356a\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") "
Jul 07 13:15:03.483083 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.482924    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7faa81ff-5e22-4eb3-b312-0db533ca356a-maas-api-tls\") pod \"7faa81ff-5e22-4eb3-b312-0db533ca356a\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") "
Jul 07 13:15:03.483083 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.483031    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7faa81ff-5e22-4eb3-b312-0db533ca356a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7faa81ff-5e22-4eb3-b312-0db533ca356a" (UID: "7faa81ff-5e22-4eb3-b312-0db533ca356a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:15:03.483268 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.483101    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7z7b\" (UniqueName: \"kubernetes.io/projected/7faa81ff-5e22-4eb3-b312-0db533ca356a-kube-api-access-r7z7b\") pod \"7faa81ff-5e22-4eb3-b312-0db533ca356a\" (UID: \"7faa81ff-5e22-4eb3-b312-0db533ca356a\") "
Jul 07 13:15:03.483386 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.483355    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7faa81ff-5e22-4eb3-b312-0db533ca356a-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:03.485336 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.485305    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7faa81ff-5e22-4eb3-b312-0db533ca356a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7faa81ff-5e22-4eb3-b312-0db533ca356a" (UID: "7faa81ff-5e22-4eb3-b312-0db533ca356a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:15:03.485668 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.485641    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7faa81ff-5e22-4eb3-b312-0db533ca356a-kube-api-access-r7z7b" (OuterVolumeSpecName: "kube-api-access-r7z7b") pod "7faa81ff-5e22-4eb3-b312-0db533ca356a" (UID: "7faa81ff-5e22-4eb3-b312-0db533ca356a"). InnerVolumeSpecName "kube-api-access-r7z7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:03.584057 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.584025    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7faa81ff-5e22-4eb3-b312-0db533ca356a-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:03.584057 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.584052    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7z7b\" (UniqueName: \"kubernetes.io/projected/7faa81ff-5e22-4eb3-b312-0db533ca356a-kube-api-access-r7z7b\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:03.842332 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.842292    2579 generic.go:356] "Generic (PLEG): container finished" podID="9683d98c-40f5-419a-8b27-ea8ce4ec4756" containerID="373f31658027c546aa497a96d069f827e78248fa06cf622ad5858bdeb5f5eb69" exitCode=6
Jul 07 13:15:03.842570 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.842376    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t" event={"ID":"9683d98c-40f5-419a-8b27-ea8ce4ec4756","Type":"ContainerDied","Data":"373f31658027c546aa497a96d069f827e78248fa06cf622ad5858bdeb5f5eb69"}
Jul 07 13:15:03.843735 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.843711    2579 generic.go:356] "Generic (PLEG): container finished" podID="7faa81ff-5e22-4eb3-b312-0db533ca356a" containerID="590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f" exitCode=0
Jul 07 13:15:03.843836 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.843752    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"
Jul 07 13:15:03.843836 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.843772    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7" event={"ID":"7faa81ff-5e22-4eb3-b312-0db533ca356a","Type":"ContainerDied","Data":"590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f"}
Jul 07 13:15:03.843836 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.843813    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7" event={"ID":"7faa81ff-5e22-4eb3-b312-0db533ca356a","Type":"ContainerDied","Data":"ad688c534d2120665a565f7b2ea4baeaff2bc467822e4b3aadc938fcd45aa48f"}
Jul 07 13:15:03.843836 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.843838    2579 scope.go:117] "RemoveContainer" containerID="590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f"
Jul 07 13:15:03.853506 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.853371    2579 scope.go:117] "RemoveContainer" containerID="590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f"
Jul 07 13:15:03.853895 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:15:03.853870    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f\": container with ID starting with 590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f not found: ID does not exist" containerID="590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f"
Jul 07 13:15:03.853970 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.853905    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f"} err="failed to get container status \"590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f\": rpc error: code = NotFound desc = could not find container \"590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f\": container with ID starting with 590b6d9b67e1bad9f520c13f71c13faac89625c4bb7f0dfaf75a8df3f399038f not found: ID does not exist"
Jul 07 13:15:03.872209 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.872178    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"]
Jul 07 13:15:03.876387 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.876356    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e2f0d5-5fcd6b7d5-prmb7"]
Jul 07 13:15:03.988374 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:03.988344    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:04.088410 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.088366    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn4j2\" (UniqueName: \"kubernetes.io/projected/9683d98c-40f5-419a-8b27-ea8ce4ec4756-kube-api-access-nn4j2\") pod \"9683d98c-40f5-419a-8b27-ea8ce4ec4756\" (UID: \"9683d98c-40f5-419a-8b27-ea8ce4ec4756\") "
Jul 07 13:15:04.090551 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.090531    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9683d98c-40f5-419a-8b27-ea8ce4ec4756-kube-api-access-nn4j2" (OuterVolumeSpecName: "kube-api-access-nn4j2") pod "9683d98c-40f5-419a-8b27-ea8ce4ec4756" (UID: "9683d98c-40f5-419a-8b27-ea8ce4ec4756"). InnerVolumeSpecName "kube-api-access-nn4j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:04.189904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.189842    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn4j2\" (UniqueName: \"kubernetes.io/projected/9683d98c-40f5-419a-8b27-ea8ce4ec4756-kube-api-access-nn4j2\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:04.519496 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.519399    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7faa81ff-5e22-4eb3-b312-0db533ca356a" path="/var/lib/kubelet/pods/7faa81ff-5e22-4eb3-b312-0db533ca356a/volumes"
Jul 07 13:15:04.851274 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.851241    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"
Jul 07 13:15:04.851499 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.851241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t" event={"ID":"9683d98c-40f5-419a-8b27-ea8ce4ec4756","Type":"ContainerDied","Data":"9676d94dc6bbc7d31e60fbfe22baf7a600ebe77569949640238b605f4e738ef3"}
Jul 07 13:15:04.851499 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.851407    2579 scope.go:117] "RemoveContainer" containerID="373f31658027c546aa497a96d069f827e78248fa06cf622ad5858bdeb5f5eb69"
Jul 07 13:15:04.866013 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.865980    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"]
Jul 07 13:15:04.869863 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:04.869830    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e2f0d5-29723835-wnj8t"]
Jul 07 13:15:06.064221 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:06.064185    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:15:06.518186 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:06.518101    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9683d98c-40f5-419a-8b27-ea8ce4ec4756" path="/var/lib/kubelet/pods/9683d98c-40f5-419a-8b27-ea8ce4ec4756/volumes"
Jul 07 13:15:13.093654 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.093616    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"]
Jul 07 13:15:13.094157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.094090    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9683d98c-40f5-419a-8b27-ea8ce4ec4756" containerName="cleanup"
Jul 07 13:15:13.094157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.094109    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9683d98c-40f5-419a-8b27-ea8ce4ec4756" containerName="cleanup"
Jul 07 13:15:13.094157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.094138    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7faa81ff-5e22-4eb3-b312-0db533ca356a" containerName="maas-api"
Jul 07 13:15:13.094157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.094147    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7faa81ff-5e22-4eb3-b312-0db533ca356a" containerName="maas-api"
Jul 07 13:15:13.094373 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.094227    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7faa81ff-5e22-4eb3-b312-0db533ca356a" containerName="maas-api"
Jul 07 13:15:13.094373 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.094240    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9683d98c-40f5-419a-8b27-ea8ce4ec4756" containerName="cleanup"
Jul 07 13:15:13.098908 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.098886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.101052 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.101025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8ff4d1-serving-cert\""
Jul 07 13:15:13.101202 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.101057    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:15:13.101202 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.101092    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:15:13.101561 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.101545    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:15:13.106958 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.106932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"]
Jul 07 13:15:13.170403 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.170366    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzscr\" (UniqueName: \"kubernetes.io/projected/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-kube-api-access-kzscr\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.170403 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.170402    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-maas-api-tls\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.170661 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.170439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.271635 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.271590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.271828 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.271693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzscr\" (UniqueName: \"kubernetes.io/projected/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-kube-api-access-kzscr\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.271828 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.271725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-maas-api-tls\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.272385 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.272364    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.274254 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.274235    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-maas-api-tls\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.283215 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.283183    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzscr\" (UniqueName: \"kubernetes.io/projected/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-kube-api-access-kzscr\") pod \"maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.412844 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.412744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:13.552876 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.552851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"]
Jul 07 13:15:13.555040 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:15:13.555010    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6ae21e_a7ad_4029_b8fc_65dc1d587791.slice/crio-c0b6b94d5ad5d25cce4ad173b84baab59dc5039622cbbd1a384acf47ace45743 WatchSource:0}: Error finding container c0b6b94d5ad5d25cce4ad173b84baab59dc5039622cbbd1a384acf47ace45743: Status 404 returned error can't find the container with id c0b6b94d5ad5d25cce4ad173b84baab59dc5039622cbbd1a384acf47ace45743
Jul 07 13:15:13.896641 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:13.896611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98" event={"ID":"3f6ae21e-a7ad-4029-b8fc-65dc1d587791","Type":"ContainerStarted","Data":"c0b6b94d5ad5d25cce4ad173b84baab59dc5039622cbbd1a384acf47ace45743"}
Jul 07 13:15:14.902696 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:14.902657    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98" event={"ID":"3f6ae21e-a7ad-4029-b8fc-65dc1d587791","Type":"ContainerStarted","Data":"eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b"}
Jul 07 13:15:14.903106 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:14.902772    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:14.918501 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:14.918455    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98" podStartSLOduration=1.441095814 podStartE2EDuration="1.918440152s" totalImagesPullingTime="477.344338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:14.917399235 +0000 UTC m=+1982.918266155" watchObservedRunningTime="2026-07-07 13:15:14.918440152 +0000 UTC m=+1982.919307070"
Jul 07 13:15:20.471167 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:20.471120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:15:20.913960 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:20.913919    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:21.176606 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.176495    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"]
Jul 07 13:15:21.180229 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.180207    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.182102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.182085    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8e00d7-serving-cert\""
Jul 07 13:15:21.189099 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.189074    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"]
Jul 07 13:15:21.241667 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.241632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4262472-c84a-43ab-995f-5fdfe72b5ab0-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.241841 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.241675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhmqh\" (UniqueName: \"kubernetes.io/projected/c4262472-c84a-43ab-995f-5fdfe72b5ab0-kube-api-access-nhmqh\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.241841 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.241737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4262472-c84a-43ab-995f-5fdfe72b5ab0-maas-api-tls\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.342861 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.342824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhmqh\" (UniqueName: \"kubernetes.io/projected/c4262472-c84a-43ab-995f-5fdfe72b5ab0-kube-api-access-nhmqh\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.343027 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.342893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4262472-c84a-43ab-995f-5fdfe72b5ab0-maas-api-tls\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.343027 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.342954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4262472-c84a-43ab-995f-5fdfe72b5ab0-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.343748 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.343726    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4262472-c84a-43ab-995f-5fdfe72b5ab0-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.345472 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.345452    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4262472-c84a-43ab-995f-5fdfe72b5ab0-maas-api-tls\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.357596 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.357574    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhmqh\" (UniqueName: \"kubernetes.io/projected/c4262472-c84a-43ab-995f-5fdfe72b5ab0-kube-api-access-nhmqh\") pod \"maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.493862 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.493767    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:21.632155 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.632130    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"]
Jul 07 13:15:21.634397 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:15:21.634365    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4262472_c84a_43ab_995f_5fdfe72b5ab0.slice/crio-568171ab17a7887ecd7a570c5295d16ad4bbf675dfedd8307df380078e24e9d6 WatchSource:0}: Error finding container 568171ab17a7887ecd7a570c5295d16ad4bbf675dfedd8307df380078e24e9d6: Status 404 returned error can't find the container with id 568171ab17a7887ecd7a570c5295d16ad4bbf675dfedd8307df380078e24e9d6
Jul 07 13:15:21.935692 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:21.935653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5" event={"ID":"c4262472-c84a-43ab-995f-5fdfe72b5ab0","Type":"ContainerStarted","Data":"568171ab17a7887ecd7a570c5295d16ad4bbf675dfedd8307df380078e24e9d6"}
Jul 07 13:15:22.947882 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:22.947841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5" event={"ID":"c4262472-c84a-43ab-995f-5fdfe72b5ab0","Type":"ContainerStarted","Data":"e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83"}
Jul 07 13:15:22.948383 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:22.947981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:28.959486 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:28.959455    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:28.985324 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:28.985268    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5" podStartSLOduration=7.603533379 podStartE2EDuration="7.985252257s" totalImagesPullingTime="381.718878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:22.965976916 +0000 UTC m=+1990.966843836" watchObservedRunningTime="2026-07-07 13:15:28.985252257 +0000 UTC m=+1996.986119176"
Jul 07 13:15:34.708773 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:34.708726    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"]
Jul 07 13:15:34.709284 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:34.708979    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98" podUID="3f6ae21e-a7ad-4029-b8fc-65dc1d587791" containerName="maas-api" containerID="cri-o://eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b" gracePeriod=30
Jul 07 13:15:34.959016 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:34.958949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:15:34.960985 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:34.960962    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:35.007619 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.007578    2579 generic.go:356] "Generic (PLEG): container finished" podID="3f6ae21e-a7ad-4029-b8fc-65dc1d587791" containerID="eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b" exitCode=0
Jul 07 13:15:35.007794 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.007627    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"
Jul 07 13:15:35.007794 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.007646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98" event={"ID":"3f6ae21e-a7ad-4029-b8fc-65dc1d587791","Type":"ContainerDied","Data":"eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b"}
Jul 07 13:15:35.007794 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.007698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98" event={"ID":"3f6ae21e-a7ad-4029-b8fc-65dc1d587791","Type":"ContainerDied","Data":"c0b6b94d5ad5d25cce4ad173b84baab59dc5039622cbbd1a384acf47ace45743"}
Jul 07 13:15:35.007794 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.007728    2579 scope.go:117] "RemoveContainer" containerID="eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b"
Jul 07 13:15:35.016310 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.016289    2579 scope.go:117] "RemoveContainer" containerID="eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b"
Jul 07 13:15:35.016595 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:15:35.016574    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b\": container with ID starting with eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b not found: ID does not exist" containerID="eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b"
Jul 07 13:15:35.016646 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.016607    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b"} err="failed to get container status \"eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b\": rpc error: code = NotFound desc = could not find container \"eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b\": container with ID starting with eba36418600407ec8f2d5ccfc34207d09c0357386e22e083379044863bb0736b not found: ID does not exist"
Jul 07 13:15:35.060443 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.060408    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzscr\" (UniqueName: \"kubernetes.io/projected/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-kube-api-access-kzscr\") pod \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") "
Jul 07 13:15:35.060663 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.060510    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-openshift-service-ca\") pod \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") "
Jul 07 13:15:35.060663 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.060590    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-maas-api-tls\") pod \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\" (UID: \"3f6ae21e-a7ad-4029-b8fc-65dc1d587791\") "
Jul 07 13:15:35.060951 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.060923    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3f6ae21e-a7ad-4029-b8fc-65dc1d587791" (UID: "3f6ae21e-a7ad-4029-b8fc-65dc1d587791"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:15:35.062703 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.062677    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-kube-api-access-kzscr" (OuterVolumeSpecName: "kube-api-access-kzscr") pod "3f6ae21e-a7ad-4029-b8fc-65dc1d587791" (UID: "3f6ae21e-a7ad-4029-b8fc-65dc1d587791"). InnerVolumeSpecName "kube-api-access-kzscr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:35.062826 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.062805    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3f6ae21e-a7ad-4029-b8fc-65dc1d587791" (UID: "3f6ae21e-a7ad-4029-b8fc-65dc1d587791"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:15:35.161849 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.161818    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzscr\" (UniqueName: \"kubernetes.io/projected/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-kube-api-access-kzscr\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:35.161849 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.161846    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:35.161849 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.161856    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f6ae21e-a7ad-4029-b8fc-65dc1d587791-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:35.328900 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.328860    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"]
Jul 07 13:15:35.332163 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:35.332135    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ff4d1-86b6bf787b-fgr98"]
Jul 07 13:15:36.517116 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:36.517078    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6ae21e-a7ad-4029-b8fc-65dc1d587791" path="/var/lib/kubelet/pods/3f6ae21e-a7ad-4029-b8fc-65dc1d587791/volumes"
Jul 07 13:15:41.676692 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:41.676645    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"]
Jul 07 13:15:41.677353 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:41.676911    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5" podUID="c4262472-c84a-43ab-995f-5fdfe72b5ab0" containerName="maas-api" containerID="cri-o://e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83" gracePeriod=30
Jul 07 13:15:41.929391 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:41.929326    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:42.019331 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.019291    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4262472-c84a-43ab-995f-5fdfe72b5ab0-maas-api-tls\") pod \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") "
Jul 07 13:15:42.019331 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.019336    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4262472-c84a-43ab-995f-5fdfe72b5ab0-openshift-service-ca\") pod \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") "
Jul 07 13:15:42.019638 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.019387    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhmqh\" (UniqueName: \"kubernetes.io/projected/c4262472-c84a-43ab-995f-5fdfe72b5ab0-kube-api-access-nhmqh\") pod \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\" (UID: \"c4262472-c84a-43ab-995f-5fdfe72b5ab0\") "
Jul 07 13:15:42.019730 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.019704    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4262472-c84a-43ab-995f-5fdfe72b5ab0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4262472-c84a-43ab-995f-5fdfe72b5ab0" (UID: "c4262472-c84a-43ab-995f-5fdfe72b5ab0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:15:42.021608 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.021580    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4262472-c84a-43ab-995f-5fdfe72b5ab0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4262472-c84a-43ab-995f-5fdfe72b5ab0" (UID: "c4262472-c84a-43ab-995f-5fdfe72b5ab0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:15:42.021730 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.021648    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4262472-c84a-43ab-995f-5fdfe72b5ab0-kube-api-access-nhmqh" (OuterVolumeSpecName: "kube-api-access-nhmqh") pod "c4262472-c84a-43ab-995f-5fdfe72b5ab0" (UID: "c4262472-c84a-43ab-995f-5fdfe72b5ab0"). InnerVolumeSpecName "kube-api-access-nhmqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:15:42.043486 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.043450    2579 generic.go:356] "Generic (PLEG): container finished" podID="c4262472-c84a-43ab-995f-5fdfe72b5ab0" containerID="e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83" exitCode=0
Jul 07 13:15:42.043657 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.043502    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"
Jul 07 13:15:42.043657 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.043555    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5" event={"ID":"c4262472-c84a-43ab-995f-5fdfe72b5ab0","Type":"ContainerDied","Data":"e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83"}
Jul 07 13:15:42.043657 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.043612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5" event={"ID":"c4262472-c84a-43ab-995f-5fdfe72b5ab0","Type":"ContainerDied","Data":"568171ab17a7887ecd7a570c5295d16ad4bbf675dfedd8307df380078e24e9d6"}
Jul 07 13:15:42.043657 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.043634    2579 scope.go:117] "RemoveContainer" containerID="e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83"
Jul 07 13:15:42.053211 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.053192    2579 scope.go:117] "RemoveContainer" containerID="e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83"
Jul 07 13:15:42.053464 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:15:42.053436    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83\": container with ID starting with e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83 not found: ID does not exist" containerID="e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83"
Jul 07 13:15:42.053585 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.053470    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83"} err="failed to get container status \"e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83\": rpc error: code = NotFound desc = could not find container \"e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83\": container with ID starting with e2be5ef564a943e906a5a58970838e2e78d13eb17a9072d70415ddaaf009ae83 not found: ID does not exist"
Jul 07 13:15:42.066717 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.066687    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"]
Jul 07 13:15:42.070336 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.070315    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e00d7-668c5b675d-22bf5"]
Jul 07 13:15:42.120390 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.120351    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4262472-c84a-43ab-995f-5fdfe72b5ab0-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:42.120390 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.120385    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4262472-c84a-43ab-995f-5fdfe72b5ab0-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:42.120390 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.120395    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhmqh\" (UniqueName: \"kubernetes.io/projected/c4262472-c84a-43ab-995f-5fdfe72b5ab0-kube-api-access-nhmqh\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:15:42.517874 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:42.517819    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4262472-c84a-43ab-995f-5fdfe72b5ab0" path="/var/lib/kubelet/pods/c4262472-c84a-43ab-995f-5fdfe72b5ab0/volumes"
Jul 07 13:15:44.670077 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:44.670044    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:15:51.408749 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.408708    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"]
Jul 07 13:15:51.409235 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.409136    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4262472-c84a-43ab-995f-5fdfe72b5ab0" containerName="maas-api"
Jul 07 13:15:51.409235 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.409155    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4262472-c84a-43ab-995f-5fdfe72b5ab0" containerName="maas-api"
Jul 07 13:15:51.409235 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.409192    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6ae21e-a7ad-4029-b8fc-65dc1d587791" containerName="maas-api"
Jul 07 13:15:51.409235 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.409200    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6ae21e-a7ad-4029-b8fc-65dc1d587791" containerName="maas-api"
Jul 07 13:15:51.409465 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.409270    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6ae21e-a7ad-4029-b8fc-65dc1d587791" containerName="maas-api"
Jul 07 13:15:51.409465 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.409285    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4262472-c84a-43ab-995f-5fdfe72b5ab0" containerName="maas-api"
Jul 07 13:15:51.411049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.411027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.414162 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.414138    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 13:15:51.414162 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.414155    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\""
Jul 07 13:15:51.414385 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.414148    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0ebff7-serving-cert\""
Jul 07 13:15:51.415026 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.415008    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 13:15:51.426496 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.426469    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"]
Jul 07 13:15:51.493843 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.493807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65652e62-199e-498f-be20-74ab69eb773a-maas-api-tls\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.494039 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.493866    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65652e62-199e-498f-be20-74ab69eb773a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.494039 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.493924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxc4\" (UniqueName: \"kubernetes.io/projected/65652e62-199e-498f-be20-74ab69eb773a-kube-api-access-cvxc4\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.595262 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.595222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65652e62-199e-498f-be20-74ab69eb773a-maas-api-tls\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.595493 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.595353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65652e62-199e-498f-be20-74ab69eb773a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.595654 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.595594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxc4\" (UniqueName: \"kubernetes.io/projected/65652e62-199e-498f-be20-74ab69eb773a-kube-api-access-cvxc4\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.596065 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.596047    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65652e62-199e-498f-be20-74ab69eb773a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.598007 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.597988    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65652e62-199e-498f-be20-74ab69eb773a-maas-api-tls\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.605064 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.605042    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxc4\" (UniqueName: \"kubernetes.io/projected/65652e62-199e-498f-be20-74ab69eb773a-kube-api-access-cvxc4\") pod \"maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.722427 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.722317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:51.857239 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:51.857214    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"]
Jul 07 13:15:51.859803 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:15:51.859776    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65652e62_199e_498f_be20_74ab69eb773a.slice/crio-007e70d911109e56f04834ba97cb84947b43486239cc2952126cc7e2c63a2f68 WatchSource:0}: Error finding container 007e70d911109e56f04834ba97cb84947b43486239cc2952126cc7e2c63a2f68: Status 404 returned error can't find the container with id 007e70d911109e56f04834ba97cb84947b43486239cc2952126cc7e2c63a2f68
Jul 07 13:15:52.097086 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:52.097042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9" event={"ID":"65652e62-199e-498f-be20-74ab69eb773a","Type":"ContainerStarted","Data":"007e70d911109e56f04834ba97cb84947b43486239cc2952126cc7e2c63a2f68"}
Jul 07 13:15:53.103114 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:53.103072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9" event={"ID":"65652e62-199e-498f-be20-74ab69eb773a","Type":"ContainerStarted","Data":"f272f7390578caec8af147e01a8e5bfdea8763a14c293c8d66eab58f6cd80ae5"}
Jul 07 13:15:53.103661 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:53.103157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:53.121194 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:53.120934    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9" podStartSLOduration=1.791373109 podStartE2EDuration="2.12091599s" totalImagesPullingTime="329.542881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:53.120463304 +0000 UTC m=+2021.121330222" watchObservedRunningTime="2026-07-07 13:15:53.12091599 +0000 UTC m=+2021.121782910"
Jul 07 13:15:58.867934 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:58.867895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:15:59.113850 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.113824    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:15:59.359721 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.359687    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"]
Jul 07 13:15:59.361883 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.361859    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.364763 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.364735    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e80c89-serving-cert\""
Jul 07 13:15:59.382262 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.382235    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"]
Jul 07 13:15:59.462632 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.462591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d1533b-0d7e-4a23-826b-a07d6c9881cf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.462825 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.462656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.462825 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.462707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9mwj\" (UniqueName: \"kubernetes.io/projected/10d1533b-0d7e-4a23-826b-a07d6c9881cf-kube-api-access-l9mwj\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.563284 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.563232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d1533b-0d7e-4a23-826b-a07d6c9881cf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.563505 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.563324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.563505 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.563367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9mwj\" (UniqueName: \"kubernetes.io/projected/10d1533b-0d7e-4a23-826b-a07d6c9881cf-kube-api-access-l9mwj\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.563505 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:15:59.563470    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-serving-cert: secret "maas-api-e2e-shared-b-e80c89-serving-cert" not found
Jul 07 13:15:59.563745 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:15:59.563567    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls podName:10d1533b-0d7e-4a23-826b-a07d6c9881cf nodeName:}" failed. No retries permitted until 2026-07-07 13:16:00.06354582 +0000 UTC m=+2028.064412723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls") pod "maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" (UID: "10d1533b-0d7e-4a23-826b-a07d6c9881cf") : secret "maas-api-e2e-shared-b-e80c89-serving-cert" not found
Jul 07 13:15:59.563928 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.563906    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d1533b-0d7e-4a23-826b-a07d6c9881cf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:15:59.579788 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:15:59.579751    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9mwj\" (UniqueName: \"kubernetes.io/projected/10d1533b-0d7e-4a23-826b-a07d6c9881cf-kube-api-access-l9mwj\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:16:00.068342 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:00.068301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:16:00.070823 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:00.070797    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls\") pod \"maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:16:00.272952 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:00.272913    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:16:00.428873 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:00.428847    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"]
Jul 07 13:16:00.431730 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:16:00.431697    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d1533b_0d7e_4a23_826b_a07d6c9881cf.slice/crio-687d7c310378973a19c545da2b0d6a1ed2ce21c4a1b19055e658f32fc1cde497 WatchSource:0}: Error finding container 687d7c310378973a19c545da2b0d6a1ed2ce21c4a1b19055e658f32fc1cde497: Status 404 returned error can't find the container with id 687d7c310378973a19c545da2b0d6a1ed2ce21c4a1b19055e658f32fc1cde497
Jul 07 13:16:01.143758 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:01.143726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" event={"ID":"10d1533b-0d7e-4a23-826b-a07d6c9881cf","Type":"ContainerStarted","Data":"415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367"}
Jul 07 13:16:01.144167 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:01.143777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" event={"ID":"10d1533b-0d7e-4a23-826b-a07d6c9881cf","Type":"ContainerStarted","Data":"687d7c310378973a19c545da2b0d6a1ed2ce21c4a1b19055e658f32fc1cde497"}
Jul 07 13:16:01.144167 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:01.143868    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:16:01.167112 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:01.167008    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" podStartSLOduration=1.650602719 podStartE2EDuration="2.166990109s" totalImagesPullingTime="516.38739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:01.164538163 +0000 UTC m=+2029.165405078" watchObservedRunningTime="2026-07-07 13:16:01.166990109 +0000 UTC m=+2029.167857046"
Jul 07 13:16:07.154819 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:07.154789    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:16:09.576533 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:09.576470    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:16:18.070802 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.070763    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"]
Jul 07 13:16:18.073157 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.073141    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.075823 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.075797    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0ebff7\"/\"default-dockercfg-knsqc\""
Jul 07 13:16:18.075823 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.075797    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0ebff7\"/\"kube-root-ca.crt\""
Jul 07 13:16:18.076014 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.075844    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0ebff7\"/\"openshift-service-ca.crt\""
Jul 07 13:16:18.076014 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.075797    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0ebff7\"/\"auth-test-model-0ebff7-kserve-self-signed-certs\""
Jul 07 13:16:18.083544 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.083507    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"]
Jul 07 13:16:18.230698 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.230661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-dshm\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.230698 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.230705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-home\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.230942 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.230727    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nhg2\" (UniqueName: \"kubernetes.io/projected/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-kube-api-access-5nhg2\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.230942 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.230758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-model-cache\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.230942 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.230778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tls-certs\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.230942 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.230842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tmp-dir\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332188 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nhg2\" (UniqueName: \"kubernetes.io/projected/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-kube-api-access-5nhg2\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332188 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-model-cache\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332188 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tls-certs\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332588 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tmp-dir\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332588 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-dshm\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332588 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-home\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332740 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332615    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-home\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332740 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332614    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-model-cache\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.332740 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.332655    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tmp-dir\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.334714 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.334688    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-dshm\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.334892 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.334874    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tls-certs\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.339750 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.339727    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nhg2\" (UniqueName: \"kubernetes.io/projected/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-kube-api-access-5nhg2\") pod \"auth-test-model-0ebff7-kserve-67494b8f87-22jrr\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") " pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.382951 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.382914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:18.524745 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:18.524718    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"]
Jul 07 13:16:18.527068 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:16:18.527040    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9222d5d5_64b3_4bbb_9f80_9e0f94cb64f1.slice/crio-4dae192acb36245b4b24dc665735123d1c6cd4f85e424700c35f5c46805fee40 WatchSource:0}: Error finding container 4dae192acb36245b4b24dc665735123d1c6cd4f85e424700c35f5c46805fee40: Status 404 returned error can't find the container with id 4dae192acb36245b4b24dc665735123d1c6cd4f85e424700c35f5c46805fee40
Jul 07 13:16:19.237462 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:19.237423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr" event={"ID":"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1","Type":"ContainerStarted","Data":"05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2"}
Jul 07 13:16:19.237462 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:19.237468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr" event={"ID":"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1","Type":"ContainerStarted","Data":"4dae192acb36245b4b24dc665735123d1c6cd4f85e424700c35f5c46805fee40"}
Jul 07 13:16:19.256233 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:19.256178    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr" podStartSLOduration=1.25616133 podStartE2EDuration="1.25616133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:19.253365099 +0000 UTC m=+2047.254232017" watchObservedRunningTime="2026-07-07 13:16:19.25616133 +0000 UTC m=+2047.257028248"
Jul 07 13:16:21.188350 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:21.188308    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:16:28.383197 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:28.383159    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:28.383667 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:28.383211    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:28.395902 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:28.395876    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:29.295955 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:29.295921    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:16:42.273938 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.273840    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"]
Jul 07 13:16:42.276688 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.276670    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.279494 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.279454    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"auth-test-model-e80c89-kserve-self-signed-certs\""
Jul 07 13:16:42.279494 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.279490    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"openshift-service-ca.crt\""
Jul 07 13:16:42.279738 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.279490    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"default-dockercfg-z27wl\""
Jul 07 13:16:42.279738 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.279471    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"kube-root-ca.crt\""
Jul 07 13:16:42.286819 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.286796    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"]
Jul 07 13:16:42.337987 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.337943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-tmp-dir\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.338175 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.337994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcf8p\" (UniqueName: \"kubernetes.io/projected/e6980d55-f36a-40af-b674-3e34f4805827-kube-api-access-xcf8p\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.338175 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.338077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-model-cache\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.338276 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.338176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-dshm\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.338276 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.338202    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-home\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.338276 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.338226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6980d55-f36a-40af-b674-3e34f4805827-tls-certs\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.438829 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.438772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-dshm\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.438829 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.438831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-home\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439118 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.438858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6980d55-f36a-40af-b674-3e34f4805827-tls-certs\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439118 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.438898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-tmp-dir\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439118 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.438923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcf8p\" (UniqueName: \"kubernetes.io/projected/e6980d55-f36a-40af-b674-3e34f4805827-kube-api-access-xcf8p\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439269 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.439135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-model-cache\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439325 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.439298    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-tmp-dir\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439366 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.439347    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-home\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.439481 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.439463    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-model-cache\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.441198 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.441180    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-dshm\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.441387 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.441369    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6980d55-f36a-40af-b674-3e34f4805827-tls-certs\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.447127 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.447099    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcf8p\" (UniqueName: \"kubernetes.io/projected/e6980d55-f36a-40af-b674-3e34f4805827-kube-api-access-xcf8p\") pod \"auth-test-model-e80c89-kserve-5fbfbbdd67-2p577\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") " pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.587249 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.587213    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:42.723191 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:42.723151    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"]
Jul 07 13:16:42.726238 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:16:42.726203    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6980d55_f36a_40af_b674_3e34f4805827.slice/crio-fdfac59fc90185556cdb0ee539c31c75b903764d981fdd2e506c0b0158d69295 WatchSource:0}: Error finding container fdfac59fc90185556cdb0ee539c31c75b903764d981fdd2e506c0b0158d69295: Status 404 returned error can't find the container with id fdfac59fc90185556cdb0ee539c31c75b903764d981fdd2e506c0b0158d69295
Jul 07 13:16:43.355680 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:43.355645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" event={"ID":"e6980d55-f36a-40af-b674-3e34f4805827","Type":"ContainerStarted","Data":"472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7"}
Jul 07 13:16:43.355680 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:43.355686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" event={"ID":"e6980d55-f36a-40af-b674-3e34f4805827","Type":"ContainerStarted","Data":"fdfac59fc90185556cdb0ee539c31c75b903764d981fdd2e506c0b0158d69295"}
Jul 07 13:16:43.373311 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:43.373253    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" podStartSLOduration=1.373233726 podStartE2EDuration="1.373233726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:43.371254162 +0000 UTC m=+2071.372121083" watchObservedRunningTime="2026-07-07 13:16:43.373233726 +0000 UTC m=+2071.374100703"
Jul 07 13:16:46.269079 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:46.269034    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:16:52.587576 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:52.587531    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:52.587576 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:52.587586    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:52.600542 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:52.600494    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:16:53.415056 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:16:53.415028    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:17:00.201942 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:17:00.201908    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:17:17.641614 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:17:17.641574    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:17:30.671791 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:17:30.671754    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:17:48.881550 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:17:48.881502    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:18:04.280097 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:04.279998    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:18:21.978839 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:21.978805    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:18:37.249883 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:37.249844    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"]
Jul 07 13:18:38.175321 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:38.175278    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"]
Jul 07 13:18:39.682571 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:39.682537    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:18:40.969481 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:40.969451    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"]
Jul 07 13:18:40.972773 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:40.972755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:40.974796 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:40.974776    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0ebff7\"/\"sub-test-model-0ebff7-kserve-self-signed-certs\""
Jul 07 13:18:40.982997 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:40.982975    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"]
Jul 07 13:18:41.043441 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.043421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tmp-dir\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.043569 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.043446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whl4g\" (UniqueName: \"kubernetes.io/projected/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-kube-api-access-whl4g\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.043569 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.043486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-dshm\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.043672 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.043571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-model-cache\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.043672 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.043626    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-home\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.043672 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.043644    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tls-certs\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.144726 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.144698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-home\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.144844 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.144733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tls-certs\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.144844 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.144774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tmp-dir\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.144844 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.144798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whl4g\" (UniqueName: \"kubernetes.io/projected/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-kube-api-access-whl4g\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.145014 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.144845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-dshm\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.145014 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.144899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-model-cache\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.145143 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.145121    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-home\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.145209 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.145163    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tmp-dir\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.145291 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.145271    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-model-cache\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.147028 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.147008    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-dshm\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.147197 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.147182    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tls-certs\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.152270 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.152247    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whl4g\" (UniqueName: \"kubernetes.io/projected/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-kube-api-access-whl4g\") pod \"sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") " pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.282561 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.282504    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:41.418035 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.418009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"]
Jul 07 13:18:41.419154 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:18:41.419127    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f1c6790_1880_45c5_8b2b_af8aa41f75f6.slice/crio-6c8d9a4380acf19e08452befd54772d4676c48821c07e0a974c9fdb9fbf6e36c WatchSource:0}: Error finding container 6c8d9a4380acf19e08452befd54772d4676c48821c07e0a974c9fdb9fbf6e36c: Status 404 returned error can't find the container with id 6c8d9a4380acf19e08452befd54772d4676c48821c07e0a974c9fdb9fbf6e36c
Jul 07 13:18:41.946470 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.946431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf" event={"ID":"0f1c6790-1880-45c5-8b2b-af8aa41f75f6","Type":"ContainerStarted","Data":"7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831"}
Jul 07 13:18:41.946470 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.946472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf" event={"ID":"0f1c6790-1880-45c5-8b2b-af8aa41f75f6","Type":"ContainerStarted","Data":"6c8d9a4380acf19e08452befd54772d4676c48821c07e0a974c9fdb9fbf6e36c"}
Jul 07 13:18:41.963145 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:41.963098    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf" podStartSLOduration=1.963083497 podStartE2EDuration="1.963083497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:41.961049807 +0000 UTC m=+2189.961916729" watchObservedRunningTime="2026-07-07 13:18:41.963083497 +0000 UTC m=+2189.963950416"
Jul 07 13:18:51.282838 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:51.282806    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:51.283329 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:51.282889    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:51.295539 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:51.295500    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:52.004772 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.004750    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:18:52.269014 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.268902    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr" podUID="9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" containerName="main" containerID="cri-o://05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2" gracePeriod=45
Jul 07 13:18:52.507597 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.507576    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:18:52.632187 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632165    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-home\") pod \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") "
Jul 07 13:18:52.632319 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632196    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nhg2\" (UniqueName: \"kubernetes.io/projected/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-kube-api-access-5nhg2\") pod \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") "
Jul 07 13:18:52.632319 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632228    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tls-certs\") pod \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") "
Jul 07 13:18:52.632319 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632260    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-model-cache\") pod \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") "
Jul 07 13:18:52.632319 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632297    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-dshm\") pod \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") "
Jul 07 13:18:52.632319 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632312    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tmp-dir\") pod \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\" (UID: \"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1\") "
Jul 07 13:18:52.632610 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632413    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-home" (OuterVolumeSpecName: "home") pod "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" (UID: "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:52.632610 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632510    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" (UID: "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:52.632723 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.632697    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" (UID: "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:52.634381 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.634337    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-dshm" (OuterVolumeSpecName: "dshm") pod "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" (UID: "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:52.634479 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.634385    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" (UID: "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:18:52.634479 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.634437    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-kube-api-access-5nhg2" (OuterVolumeSpecName: "kube-api-access-5nhg2") pod "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" (UID: "9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1"). InnerVolumeSpecName "kube-api-access-5nhg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:18:52.733753 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.733726    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:52.733753 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.733751    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nhg2\" (UniqueName: \"kubernetes.io/projected/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-kube-api-access-5nhg2\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:52.733931 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.733760    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:52.733931 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.733770    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:52.733931 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.733777    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:52.733931 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:52.733785    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:53.001143 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.001050    2579 generic.go:356] "Generic (PLEG): container finished" podID="9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" containerID="05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2" exitCode=0
Jul 07 13:18:53.001143 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.001091    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"
Jul 07 13:18:53.001461 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.001138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr" event={"ID":"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1","Type":"ContainerDied","Data":"05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2"}
Jul 07 13:18:53.001461 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.001195    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr" event={"ID":"9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1","Type":"ContainerDied","Data":"4dae192acb36245b4b24dc665735123d1c6cd4f85e424700c35f5c46805fee40"}
Jul 07 13:18:53.001461 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.001220    2579 scope.go:117] "RemoveContainer" containerID="05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2"
Jul 07 13:18:53.012291 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.012276    2579 scope.go:117] "RemoveContainer" containerID="05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2"
Jul 07 13:18:53.012578 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:18:53.012559    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2\": container with ID starting with 05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2 not found: ID does not exist" containerID="05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2"
Jul 07 13:18:53.012662 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.012591    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2"} err="failed to get container status \"05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2\": rpc error: code = NotFound desc = could not find container \"05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2\": container with ID starting with 05bb44bd77ab51999090f496790ba273924ef914db20a37a2930b0e9d2985ec2 not found: ID does not exist"
Jul 07 13:18:53.023392 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.023367    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"]
Jul 07 13:18:53.025609 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.025587    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/auth-test-model-0ebff7-kserve-67494b8f87-22jrr"]
Jul 07 13:18:53.195320 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.195288    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" podUID="e6980d55-f36a-40af-b674-3e34f4805827" containerName="main" containerID="cri-o://472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7" gracePeriod=45
Jul 07 13:18:53.434926 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.434901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:18:53.540645 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540569    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-tmp-dir\") pod \"e6980d55-f36a-40af-b674-3e34f4805827\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") "
Jul 07 13:18:53.540645 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcf8p\" (UniqueName: \"kubernetes.io/projected/e6980d55-f36a-40af-b674-3e34f4805827-kube-api-access-xcf8p\") pod \"e6980d55-f36a-40af-b674-3e34f4805827\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") "
Jul 07 13:18:53.540645 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540633    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6980d55-f36a-40af-b674-3e34f4805827-tls-certs\") pod \"e6980d55-f36a-40af-b674-3e34f4805827\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") "
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540678    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-dshm\") pod \"e6980d55-f36a-40af-b674-3e34f4805827\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") "
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540709    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-model-cache\") pod \"e6980d55-f36a-40af-b674-3e34f4805827\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") "
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-home\") pod \"e6980d55-f36a-40af-b674-3e34f4805827\" (UID: \"e6980d55-f36a-40af-b674-3e34f4805827\") "
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.540843    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6980d55-f36a-40af-b674-3e34f4805827" (UID: "e6980d55-f36a-40af-b674-3e34f4805827"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.541005    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.541003    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6980d55-f36a-40af-b674-3e34f4805827" (UID: "e6980d55-f36a-40af-b674-3e34f4805827"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:53.541154 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.541019    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-home" (OuterVolumeSpecName: "home") pod "e6980d55-f36a-40af-b674-3e34f4805827" (UID: "e6980d55-f36a-40af-b674-3e34f4805827"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:53.542852 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.542825    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6980d55-f36a-40af-b674-3e34f4805827-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6980d55-f36a-40af-b674-3e34f4805827" (UID: "e6980d55-f36a-40af-b674-3e34f4805827"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:18:53.542962 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.542878    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-dshm" (OuterVolumeSpecName: "dshm") pod "e6980d55-f36a-40af-b674-3e34f4805827" (UID: "e6980d55-f36a-40af-b674-3e34f4805827"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:18:53.542962 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.542895    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6980d55-f36a-40af-b674-3e34f4805827-kube-api-access-xcf8p" (OuterVolumeSpecName: "kube-api-access-xcf8p") pod "e6980d55-f36a-40af-b674-3e34f4805827" (UID: "e6980d55-f36a-40af-b674-3e34f4805827"). InnerVolumeSpecName "kube-api-access-xcf8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:18:53.641729 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.641705    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:53.641729 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.641724    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:53.641729 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.641735    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6980d55-f36a-40af-b674-3e34f4805827-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:53.641930 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.641744    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcf8p\" (UniqueName: \"kubernetes.io/projected/e6980d55-f36a-40af-b674-3e34f4805827-kube-api-access-xcf8p\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:53.641930 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:53.641752    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6980d55-f36a-40af-b674-3e34f4805827-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:18:54.008150 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.008113    2579 generic.go:356] "Generic (PLEG): container finished" podID="e6980d55-f36a-40af-b674-3e34f4805827" containerID="472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7" exitCode=0
Jul 07 13:18:54.008351 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.008157    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"
Jul 07 13:18:54.008351 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.008196    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" event={"ID":"e6980d55-f36a-40af-b674-3e34f4805827","Type":"ContainerDied","Data":"472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7"}
Jul 07 13:18:54.008351 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.008238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" event={"ID":"e6980d55-f36a-40af-b674-3e34f4805827","Type":"ContainerDied","Data":"fdfac59fc90185556cdb0ee539c31c75b903764d981fdd2e506c0b0158d69295"}
Jul 07 13:18:54.008351 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.008255    2579 scope.go:117] "RemoveContainer" containerID="472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7"
Jul 07 13:18:54.016783 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.016766    2579 scope.go:117] "RemoveContainer" containerID="472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7"
Jul 07 13:18:54.017015 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:18:54.016998    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7\": container with ID starting with 472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7 not found: ID does not exist" containerID="472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7"
Jul 07 13:18:54.017079 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.017021    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7"} err="failed to get container status \"472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7\": rpc error: code = NotFound desc = could not find container \"472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7\": container with ID starting with 472390dfb38c457c5fee8acc83ac8ce5c0d9a135ea301a094195ed5bf4a9d5d7 not found: ID does not exist"
Jul 07 13:18:54.028346 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.028325    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"]
Jul 07 13:18:54.034059 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.034040    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577"]
Jul 07 13:18:54.524732 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.524698    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" path="/var/lib/kubelet/pods/9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1/volumes"
Jul 07 13:18:54.525105 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:54.525089    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6980d55-f36a-40af-b674-3e34f4805827" path="/var/lib/kubelet/pods/e6980d55-f36a-40af-b674-3e34f4805827/volumes"
Jul 07 13:18:55.676756 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:55.676713    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:18:58.405463 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:18:58.405398    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-e80c89/auth-test-model-e80c89-kserve-5fbfbbdd67-2p577" podUID="e6980d55-f36a-40af-b674-3e34f4805827" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Jul 07 13:19:16.584002 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.583943    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"]
Jul 07 13:19:16.584904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.584874    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" containerName="main"
Jul 07 13:19:16.584904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.584898    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" containerName="main"
Jul 07 13:19:16.584904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.584906    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6980d55-f36a-40af-b674-3e34f4805827" containerName="main"
Jul 07 13:19:16.585089 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.584912    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6980d55-f36a-40af-b674-3e34f4805827" containerName="main"
Jul 07 13:19:16.585089 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.584965    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6980d55-f36a-40af-b674-3e34f4805827" containerName="main"
Jul 07 13:19:16.585089 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.584974    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9222d5d5-64b3-4bbb-9f80-9e0f94cb64f1" containerName="main"
Jul 07 13:19:16.589420 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.589397    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.591019 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.590990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"]
Jul 07 13:19:16.591335 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.591317    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"kube-root-ca.crt\""
Jul 07 13:19:16.591843 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.591823    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"default-dockercfg-z27wl\""
Jul 07 13:19:16.591949 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.591822    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"openshift-service-ca.crt\""
Jul 07 13:19:16.591949 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.591905    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"sub-test-model-e80c89-kserve-self-signed-certs\""
Jul 07 13:19:16.729887 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.729849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-dshm\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.729887 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.729898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-model-cache\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.730124 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.729939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tmp-dir\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.730124 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.729983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-home\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.730124 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.730006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfx2l\" (UniqueName: \"kubernetes.io/projected/57f0ff8f-a692-4b19-aa52-359650ae3c9f-kube-api-access-pfx2l\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.730124 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.730025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tls-certs\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831388 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-dshm\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831388 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-model-cache\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831640 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tmp-dir\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831640 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-home\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831640 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfx2l\" (UniqueName: \"kubernetes.io/projected/57f0ff8f-a692-4b19-aa52-359650ae3c9f-kube-api-access-pfx2l\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831793 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tls-certs\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831879 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831855    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tmp-dir\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.831932 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.831883    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-home\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.832028 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.832009    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-model-cache\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.833639 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.833622    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-dshm\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.834194 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.834140    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tls-certs\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.841629 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.841605    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfx2l\" (UniqueName: \"kubernetes.io/projected/57f0ff8f-a692-4b19-aa52-359650ae3c9f-kube-api-access-pfx2l\") pod \"sub-test-model-e80c89-kserve-5586d5d94f-cqbgr\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") " pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:16.899664 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:16.899613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:17.042301 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:17.042270    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"]
Jul 07 13:19:17.044113 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:19:17.044081    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57f0ff8f_a692_4b19_aa52_359650ae3c9f.slice/crio-1686318250a76fc99bf178de8f96bf5a96d9c4895c12186a915e7c3bd15f7ff3 WatchSource:0}: Error finding container 1686318250a76fc99bf178de8f96bf5a96d9c4895c12186a915e7c3bd15f7ff3: Status 404 returned error can't find the container with id 1686318250a76fc99bf178de8f96bf5a96d9c4895c12186a915e7c3bd15f7ff3
Jul 07 13:19:17.131120 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:17.131084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr" event={"ID":"57f0ff8f-a692-4b19-aa52-359650ae3c9f","Type":"ContainerStarted","Data":"f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0"}
Jul 07 13:19:17.131326 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:17.131129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr" event={"ID":"57f0ff8f-a692-4b19-aa52-359650ae3c9f","Type":"ContainerStarted","Data":"1686318250a76fc99bf178de8f96bf5a96d9c4895c12186a915e7c3bd15f7ff3"}
Jul 07 13:19:17.161368 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:17.161315    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr" podStartSLOduration=1.161298 podStartE2EDuration="1.161298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:19:17.16074273 +0000 UTC m=+2225.161609650" watchObservedRunningTime="2026-07-07 13:19:17.161298 +0000 UTC m=+2225.162164960"
Jul 07 13:19:21.270133 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:21.270098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:19:26.900588 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:26.900539    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:26.901109 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:26.900605    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:26.913351 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:26.913317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:27.201047 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:27.200947    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:19:36.872594 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:36.872498    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:19:54.656084 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:54.656043    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"]
Jul 07 13:19:55.641380 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:55.641346    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"]
Jul 07 13:19:58.879624 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:19:58.879587    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:20:02.974173 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:02.974139    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"]
Jul 07 13:20:02.977604 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:02.977585    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:02.979695 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:02.979665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0ebff7\"/\"rate-test-model-0ebff7-kserve-self-signed-certs\""
Jul 07 13:20:02.989208 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:02.989181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"]
Jul 07 13:20:03.018489 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.018455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5btk\" (UniqueName: \"kubernetes.io/projected/a0beb99b-2a7c-4fbb-b35e-26c231085c03-kube-api-access-h5btk\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.018705 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.018556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-model-cache\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.018705 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.018583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-home\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.018705 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.018689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tls-certs\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.018875 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.018743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-dshm\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.018875 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.018773    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tmp-dir\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.119626 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.119587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-model-cache\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.119626 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.119629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-home\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.119881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.119681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tls-certs\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.119881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.119709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-dshm\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.119881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.119727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tmp-dir\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.119881 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.119785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5btk\" (UniqueName: \"kubernetes.io/projected/a0beb99b-2a7c-4fbb-b35e-26c231085c03-kube-api-access-h5btk\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.120127 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.120098    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-model-cache\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.120313 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.120121    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-home\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.120313 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.120189    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tmp-dir\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.122061 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.122035    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-dshm\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.122442 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.122424    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tls-certs\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.128216 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.128183    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5btk\" (UniqueName: \"kubernetes.io/projected/a0beb99b-2a7c-4fbb-b35e-26c231085c03-kube-api-access-h5btk\") pod \"rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") " pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.288327 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.288228    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:03.435171 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:03.435139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"]
Jul 07 13:20:03.437952 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:20:03.437923    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0beb99b_2a7c_4fbb_b35e_26c231085c03.slice/crio-9c9b3304e27388d264b580de9cce7838514f627feedb0a451434f109bb51f52e WatchSource:0}: Error finding container 9c9b3304e27388d264b580de9cce7838514f627feedb0a451434f109bb51f52e: Status 404 returned error can't find the container with id 9c9b3304e27388d264b580de9cce7838514f627feedb0a451434f109bb51f52e
Jul 07 13:20:04.372956 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:04.372918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6" event={"ID":"a0beb99b-2a7c-4fbb-b35e-26c231085c03","Type":"ContainerStarted","Data":"2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9"}
Jul 07 13:20:04.373347 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:04.372963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6" event={"ID":"a0beb99b-2a7c-4fbb-b35e-26c231085c03","Type":"ContainerStarted","Data":"9c9b3304e27388d264b580de9cce7838514f627feedb0a451434f109bb51f52e"}
Jul 07 13:20:04.389490 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:04.389435    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6" podStartSLOduration=2.389416413 podStartE2EDuration="2.389416413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:04.388668004 +0000 UTC m=+2272.389534923" watchObservedRunningTime="2026-07-07 13:20:04.389416413 +0000 UTC m=+2272.390283332"
Jul 07 13:20:09.675835 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.675794    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf" podUID="0f1c6790-1880-45c5-8b2b-af8aa41f75f6" containerName="main" containerID="cri-o://7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831" gracePeriod=45
Jul 07 13:20:09.927583 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.927499    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:20:09.981449 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tls-certs\") pod \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") "
Jul 07 13:20:09.981449 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981454    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-dshm\") pod \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") "
Jul 07 13:20:09.981714 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-home\") pod \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") "
Jul 07 13:20:09.981714 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981554    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whl4g\" (UniqueName: \"kubernetes.io/projected/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-kube-api-access-whl4g\") pod \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") "
Jul 07 13:20:09.981714 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-model-cache\") pod \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") "
Jul 07 13:20:09.981714 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981593    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tmp-dir\") pod \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\" (UID: \"0f1c6790-1880-45c5-8b2b-af8aa41f75f6\") "
Jul 07 13:20:09.981941 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981874    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-home" (OuterVolumeSpecName: "home") pod "0f1c6790-1880-45c5-8b2b-af8aa41f75f6" (UID: "0f1c6790-1880-45c5-8b2b-af8aa41f75f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:09.981941 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981891    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "0f1c6790-1880-45c5-8b2b-af8aa41f75f6" (UID: "0f1c6790-1880-45c5-8b2b-af8aa41f75f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:09.982044 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.981970    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0f1c6790-1880-45c5-8b2b-af8aa41f75f6" (UID: "0f1c6790-1880-45c5-8b2b-af8aa41f75f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:09.983782 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.983757    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-kube-api-access-whl4g" (OuterVolumeSpecName: "kube-api-access-whl4g") pod "0f1c6790-1880-45c5-8b2b-af8aa41f75f6" (UID: "0f1c6790-1880-45c5-8b2b-af8aa41f75f6"). InnerVolumeSpecName "kube-api-access-whl4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:20:09.983899 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.983784    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0f1c6790-1880-45c5-8b2b-af8aa41f75f6" (UID: "0f1c6790-1880-45c5-8b2b-af8aa41f75f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:20:09.983899 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:09.983816    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-dshm" (OuterVolumeSpecName: "dshm") pod "0f1c6790-1880-45c5-8b2b-af8aa41f75f6" (UID: "0f1c6790-1880-45c5-8b2b-af8aa41f75f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:10.083121 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.083077    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:10.083121 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.083113    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:10.083121 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.083127    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whl4g\" (UniqueName: \"kubernetes.io/projected/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-kube-api-access-whl4g\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:10.083372 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.083142    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:10.083372 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.083169    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:10.083372 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.083180    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f1c6790-1880-45c5-8b2b-af8aa41f75f6-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:10.405174 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.405133    2579 generic.go:356] "Generic (PLEG): container finished" podID="0f1c6790-1880-45c5-8b2b-af8aa41f75f6" containerID="7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831" exitCode=0
Jul 07 13:20:10.405355 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.405185    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"
Jul 07 13:20:10.405355 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.405222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf" event={"ID":"0f1c6790-1880-45c5-8b2b-af8aa41f75f6","Type":"ContainerDied","Data":"7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831"}
Jul 07 13:20:10.405355 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.405272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf" event={"ID":"0f1c6790-1880-45c5-8b2b-af8aa41f75f6","Type":"ContainerDied","Data":"6c8d9a4380acf19e08452befd54772d4676c48821c07e0a974c9fdb9fbf6e36c"}
Jul 07 13:20:10.405355 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.405290    2579 scope.go:117] "RemoveContainer" containerID="7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831"
Jul 07 13:20:10.415051 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.415030    2579 scope.go:117] "RemoveContainer" containerID="7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831"
Jul 07 13:20:10.415335 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:20:10.415317    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831\": container with ID starting with 7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831 not found: ID does not exist" containerID="7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831"
Jul 07 13:20:10.415411 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.415342    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831"} err="failed to get container status \"7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831\": rpc error: code = NotFound desc = could not find container \"7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831\": container with ID starting with 7812719c4bd7247c60dda58724e364988b0ae03a945712cf7e024afbe7050831 not found: ID does not exist"
Jul 07 13:20:10.426869 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.426840    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"]
Jul 07 13:20:10.430994 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.430957    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/sub-test-model-0ebff7-kserve-6b5b5fb95-4pwtf"]
Jul 07 13:20:10.517890 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.517855    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f1c6790-1880-45c5-8b2b-af8aa41f75f6" path="/var/lib/kubelet/pods/0f1c6790-1880-45c5-8b2b-af8aa41f75f6/volumes"
Jul 07 13:20:10.661847 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.661769    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr" podUID="57f0ff8f-a692-4b19-aa52-359650ae3c9f" containerName="main" containerID="cri-o://f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0" gracePeriod=45
Jul 07 13:20:10.919337 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.919277    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:20:10.991016 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.990978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-dshm\") pod \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") "
Jul 07 13:20:10.991226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991029    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfx2l\" (UniqueName: \"kubernetes.io/projected/57f0ff8f-a692-4b19-aa52-359650ae3c9f-kube-api-access-pfx2l\") pod \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") "
Jul 07 13:20:10.991226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991079    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-model-cache\") pod \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") "
Jul 07 13:20:10.991226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-home\") pod \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") "
Jul 07 13:20:10.991226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991132    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tls-certs\") pod \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") "
Jul 07 13:20:10.991226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991155    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tmp-dir\") pod \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\" (UID: \"57f0ff8f-a692-4b19-aa52-359650ae3c9f\") "
Jul 07 13:20:10.991491 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991399    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-home" (OuterVolumeSpecName: "home") pod "57f0ff8f-a692-4b19-aa52-359650ae3c9f" (UID: "57f0ff8f-a692-4b19-aa52-359650ae3c9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:10.991491 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991410    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "57f0ff8f-a692-4b19-aa52-359650ae3c9f" (UID: "57f0ff8f-a692-4b19-aa52-359650ae3c9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:10.991597 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.991488    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57f0ff8f-a692-4b19-aa52-359650ae3c9f" (UID: "57f0ff8f-a692-4b19-aa52-359650ae3c9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:10.993234 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.993204    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57f0ff8f-a692-4b19-aa52-359650ae3c9f" (UID: "57f0ff8f-a692-4b19-aa52-359650ae3c9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:20:10.993335 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.993245    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-dshm" (OuterVolumeSpecName: "dshm") pod "57f0ff8f-a692-4b19-aa52-359650ae3c9f" (UID: "57f0ff8f-a692-4b19-aa52-359650ae3c9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:20:10.993335 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:10.993249    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f0ff8f-a692-4b19-aa52-359650ae3c9f-kube-api-access-pfx2l" (OuterVolumeSpecName: "kube-api-access-pfx2l") pod "57f0ff8f-a692-4b19-aa52-359650ae3c9f" (UID: "57f0ff8f-a692-4b19-aa52-359650ae3c9f"). InnerVolumeSpecName "kube-api-access-pfx2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:20:11.092575 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.092509    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:11.092575 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.092568    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:11.092575 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.092581    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfx2l\" (UniqueName: \"kubernetes.io/projected/57f0ff8f-a692-4b19-aa52-359650ae3c9f-kube-api-access-pfx2l\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:11.092830 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.092593    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:11.092830 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.092606    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f0ff8f-a692-4b19-aa52-359650ae3c9f-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:11.092830 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.092617    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f0ff8f-a692-4b19-aa52-359650ae3c9f-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:20:11.411147 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.411114    2579 generic.go:356] "Generic (PLEG): container finished" podID="57f0ff8f-a692-4b19-aa52-359650ae3c9f" containerID="f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0" exitCode=0
Jul 07 13:20:11.411340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.411154    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"
Jul 07 13:20:11.411340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.411192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr" event={"ID":"57f0ff8f-a692-4b19-aa52-359650ae3c9f","Type":"ContainerDied","Data":"f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0"}
Jul 07 13:20:11.411340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.411231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr" event={"ID":"57f0ff8f-a692-4b19-aa52-359650ae3c9f","Type":"ContainerDied","Data":"1686318250a76fc99bf178de8f96bf5a96d9c4895c12186a915e7c3bd15f7ff3"}
Jul 07 13:20:11.411340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.411254    2579 scope.go:117] "RemoveContainer" containerID="f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0"
Jul 07 13:20:11.420739 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.420719    2579 scope.go:117] "RemoveContainer" containerID="f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0"
Jul 07 13:20:11.421022 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:20:11.421002    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0\": container with ID starting with f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0 not found: ID does not exist" containerID="f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0"
Jul 07 13:20:11.421071 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.421033    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0"} err="failed to get container status \"f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0\": rpc error: code = NotFound desc = could not find container \"f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0\": container with ID starting with f304c81a08cd16050677e3d230bb0ea5cab56b4c3ea8a02eada89293d372b4c0 not found: ID does not exist"
Jul 07 13:20:11.434618 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.434587    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"]
Jul 07 13:20:11.441082 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:11.441055    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/sub-test-model-e80c89-kserve-5586d5d94f-cqbgr"]
Jul 07 13:20:12.518101 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:12.518068    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f0ff8f-a692-4b19-aa52-359650ae3c9f" path="/var/lib/kubelet/pods/57f0ff8f-a692-4b19-aa52-359650ae3c9f/volumes"
Jul 07 13:20:13.289299 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:13.289262    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:13.289485 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:13.289316    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:13.302477 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:13.302451    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:13.434737 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:13.434703    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:20:14.480907 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:14.480872    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:20:27.576479 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.576445    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"]
Jul 07 13:20:27.576948 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.576912    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1c6790-1880-45c5-8b2b-af8aa41f75f6" containerName="main"
Jul 07 13:20:27.576948 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.576929    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1c6790-1880-45c5-8b2b-af8aa41f75f6" containerName="main"
Jul 07 13:20:27.577074 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.576949    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f0ff8f-a692-4b19-aa52-359650ae3c9f" containerName="main"
Jul 07 13:20:27.577074 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.576958    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f0ff8f-a692-4b19-aa52-359650ae3c9f" containerName="main"
Jul 07 13:20:27.577074 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.577039    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f1c6790-1880-45c5-8b2b-af8aa41f75f6" containerName="main"
Jul 07 13:20:27.577074 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.577050    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f0ff8f-a692-4b19-aa52-359650ae3c9f" containerName="main"
Jul 07 13:20:27.580104 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.580080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.584635 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.584611    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"openshift-service-ca.crt\""
Jul 07 13:20:27.585138 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.585113    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"kube-root-ca.crt\""
Jul 07 13:20:27.585254 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.585123    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"default-dockercfg-z27wl\""
Jul 07 13:20:27.585254 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.585123    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e80c89\"/\"rate-test-model-e80c89-kserve-self-signed-certs\""
Jul 07 13:20:27.591569 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.591547    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"]
Jul 07 13:20:27.742683 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.742651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l542t\" (UniqueName: \"kubernetes.io/projected/544779b9-d61f-4c1d-b429-e2e61742c107-kube-api-access-l542t\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.742683 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.742686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-tmp-dir\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.742904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.742709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-dshm\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.742904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.742835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/544779b9-d61f-4c1d-b429-e2e61742c107-tls-certs\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.742904 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.742866    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-model-cache\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.743006 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.742917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-home\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844054 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.843958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l542t\" (UniqueName: \"kubernetes.io/projected/544779b9-d61f-4c1d-b429-e2e61742c107-kube-api-access-l542t\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844054 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.843999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-tmp-dir\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844054 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-dshm\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844349 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/544779b9-d61f-4c1d-b429-e2e61742c107-tls-certs\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844349 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-model-cache\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844349 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-home\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844583 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844452    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-tmp-dir\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844583 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844475    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-home\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.844583 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.844542    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-model-cache\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.846367 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.846345    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-dshm\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.846656 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.846639    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/544779b9-d61f-4c1d-b429-e2e61742c107-tls-certs\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.852184 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.852164    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l542t\" (UniqueName: \"kubernetes.io/projected/544779b9-d61f-4c1d-b429-e2e61742c107-kube-api-access-l542t\") pod \"rate-test-model-e80c89-kserve-964f76455-7dl6n\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") " pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:27.890719 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:27.890677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:28.028368 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:28.028346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"]
Jul 07 13:20:28.030779 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:20:28.030748    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod544779b9_d61f_4c1d_b429_e2e61742c107.slice/crio-43fb19faf4c7e7f559b42dfae2eafe1b827cadc9c6c5cc1555ae8b9fd9d920f8 WatchSource:0}: Error finding container 43fb19faf4c7e7f559b42dfae2eafe1b827cadc9c6c5cc1555ae8b9fd9d920f8: Status 404 returned error can't find the container with id 43fb19faf4c7e7f559b42dfae2eafe1b827cadc9c6c5cc1555ae8b9fd9d920f8
Jul 07 13:20:28.499798 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:28.499763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n" event={"ID":"544779b9-d61f-4c1d-b429-e2e61742c107","Type":"ContainerStarted","Data":"593d332c38205727aae5dcb56ef8f566736f51e8b84a6018fce8ba8a86b4f5f6"}
Jul 07 13:20:28.499984 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:28.499807    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n" event={"ID":"544779b9-d61f-4c1d-b429-e2e61742c107","Type":"ContainerStarted","Data":"43fb19faf4c7e7f559b42dfae2eafe1b827cadc9c6c5cc1555ae8b9fd9d920f8"}
Jul 07 13:20:28.522353 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:28.522283    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n" podStartSLOduration=1.522260652 podStartE2EDuration="1.522260652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:20:28.520359654 +0000 UTC m=+2296.521226573" watchObservedRunningTime="2026-07-07 13:20:28.522260652 +0000 UTC m=+2296.523127571"
Jul 07 13:20:31.778599 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:31.778567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:20:37.891191 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:37.891138    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:37.891191 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:37.891194    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:37.904223 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:37.904195    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:38.559996 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:38.559966    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:20:52.786209 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:20:52.786172    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:21:13.472563 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:13.472508    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:21:17.468168 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:17.468128    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"]
Jul 07 13:21:18.597969 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:18.597935    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"]
Jul 07 13:21:20.579671 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:20.579633    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"]
Jul 07 13:21:20.580083 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:20.579849    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9" podUID="65652e62-199e-498f-be20-74ab69eb773a" containerName="maas-api" containerID="cri-o://f272f7390578caec8af147e01a8e5bfdea8763a14c293c8d66eab58f6cd80ae5" gracePeriod=30
Jul 07 13:21:20.760473 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:20.760439    2579 generic.go:356] "Generic (PLEG): container finished" podID="65652e62-199e-498f-be20-74ab69eb773a" containerID="f272f7390578caec8af147e01a8e5bfdea8763a14c293c8d66eab58f6cd80ae5" exitCode=0
Jul 07 13:21:20.760652 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:20.760511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9" event={"ID":"65652e62-199e-498f-be20-74ab69eb773a","Type":"ContainerDied","Data":"f272f7390578caec8af147e01a8e5bfdea8763a14c293c8d66eab58f6cd80ae5"}
Jul 07 13:21:20.833102 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:20.833035    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:21:21.004093 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.004052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvxc4\" (UniqueName: \"kubernetes.io/projected/65652e62-199e-498f-be20-74ab69eb773a-kube-api-access-cvxc4\") pod \"65652e62-199e-498f-be20-74ab69eb773a\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") "
Jul 07 13:21:21.004093 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.004097    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65652e62-199e-498f-be20-74ab69eb773a-maas-api-tls\") pod \"65652e62-199e-498f-be20-74ab69eb773a\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") "
Jul 07 13:21:21.004352 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.004131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65652e62-199e-498f-be20-74ab69eb773a-openshift-service-ca\") pod \"65652e62-199e-498f-be20-74ab69eb773a\" (UID: \"65652e62-199e-498f-be20-74ab69eb773a\") "
Jul 07 13:21:21.004646 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.004616    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65652e62-199e-498f-be20-74ab69eb773a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65652e62-199e-498f-be20-74ab69eb773a" (UID: "65652e62-199e-498f-be20-74ab69eb773a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:21:21.006269 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.006241    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65652e62-199e-498f-be20-74ab69eb773a-kube-api-access-cvxc4" (OuterVolumeSpecName: "kube-api-access-cvxc4") pod "65652e62-199e-498f-be20-74ab69eb773a" (UID: "65652e62-199e-498f-be20-74ab69eb773a"). InnerVolumeSpecName "kube-api-access-cvxc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:21:21.006269 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.006259    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65652e62-199e-498f-be20-74ab69eb773a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65652e62-199e-498f-be20-74ab69eb773a" (UID: "65652e62-199e-498f-be20-74ab69eb773a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:21:21.105646 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.105553    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvxc4\" (UniqueName: \"kubernetes.io/projected/65652e62-199e-498f-be20-74ab69eb773a-kube-api-access-cvxc4\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:21.105646 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.105586    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65652e62-199e-498f-be20-74ab69eb773a-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:21.105646 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.105595    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65652e62-199e-498f-be20-74ab69eb773a-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:21.767945 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.767917    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"
Jul 07 13:21:21.768418 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.767918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9" event={"ID":"65652e62-199e-498f-be20-74ab69eb773a","Type":"ContainerDied","Data":"007e70d911109e56f04834ba97cb84947b43486239cc2952126cc7e2c63a2f68"}
Jul 07 13:21:21.768418 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.768052    2579 scope.go:117] "RemoveContainer" containerID="f272f7390578caec8af147e01a8e5bfdea8763a14c293c8d66eab58f6cd80ae5"
Jul 07 13:21:21.790131 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.790102    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"]
Jul 07 13:21:21.796483 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:21.796454    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0ebff7-556467d5c6-b5mh9"]
Jul 07 13:21:22.517898 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:22.517861    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65652e62-199e-498f-be20-74ab69eb773a" path="/var/lib/kubelet/pods/65652e62-199e-498f-be20-74ab69eb773a/volumes"
Jul 07 13:21:28.880850 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:28.880815    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:21:32.488363 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.488322    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6" podUID="a0beb99b-2a7c-4fbb-b35e-26c231085c03" containerName="main" containerID="cri-o://2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9" gracePeriod=45
Jul 07 13:21:32.741370 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.741266    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:21:32.792536 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792492    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-model-cache\") pod \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") "
Jul 07 13:21:32.792793 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792555    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-home\") pod \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") "
Jul 07 13:21:32.792793 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792588    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tls-certs\") pod \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") "
Jul 07 13:21:32.792793 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792615    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5btk\" (UniqueName: \"kubernetes.io/projected/a0beb99b-2a7c-4fbb-b35e-26c231085c03-kube-api-access-h5btk\") pod \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") "
Jul 07 13:21:32.792793 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792665    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-dshm\") pod \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") "
Jul 07 13:21:32.792793 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tmp-dir\") pod \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\" (UID: \"a0beb99b-2a7c-4fbb-b35e-26c231085c03\") "
Jul 07 13:21:32.793067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792810    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0beb99b-2a7c-4fbb-b35e-26c231085c03" (UID: "a0beb99b-2a7c-4fbb-b35e-26c231085c03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:32.793067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792874    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-home" (OuterVolumeSpecName: "home") pod "a0beb99b-2a7c-4fbb-b35e-26c231085c03" (UID: "a0beb99b-2a7c-4fbb-b35e-26c231085c03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:32.793067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.792991    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0beb99b-2a7c-4fbb-b35e-26c231085c03" (UID: "a0beb99b-2a7c-4fbb-b35e-26c231085c03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:32.793067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.793031    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:32.793067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.793050    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:32.794847 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.794821    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-dshm" (OuterVolumeSpecName: "dshm") pod "a0beb99b-2a7c-4fbb-b35e-26c231085c03" (UID: "a0beb99b-2a7c-4fbb-b35e-26c231085c03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:32.794928 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.794903    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0beb99b-2a7c-4fbb-b35e-26c231085c03" (UID: "a0beb99b-2a7c-4fbb-b35e-26c231085c03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:21:32.794928 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.794913    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0beb99b-2a7c-4fbb-b35e-26c231085c03-kube-api-access-h5btk" (OuterVolumeSpecName: "kube-api-access-h5btk") pod "a0beb99b-2a7c-4fbb-b35e-26c231085c03" (UID: "a0beb99b-2a7c-4fbb-b35e-26c231085c03"). InnerVolumeSpecName "kube-api-access-h5btk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:21:32.828574 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.828535    2579 generic.go:356] "Generic (PLEG): container finished" podID="a0beb99b-2a7c-4fbb-b35e-26c231085c03" containerID="2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9" exitCode=0
Jul 07 13:21:32.828753 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.828580    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"
Jul 07 13:21:32.828753 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.828610    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6" event={"ID":"a0beb99b-2a7c-4fbb-b35e-26c231085c03","Type":"ContainerDied","Data":"2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9"}
Jul 07 13:21:32.828753 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.828654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6" event={"ID":"a0beb99b-2a7c-4fbb-b35e-26c231085c03","Type":"ContainerDied","Data":"9c9b3304e27388d264b580de9cce7838514f627feedb0a451434f109bb51f52e"}
Jul 07 13:21:32.828753 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.828671    2579 scope.go:117] "RemoveContainer" containerID="2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9"
Jul 07 13:21:32.837481 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.837464    2579 scope.go:117] "RemoveContainer" containerID="2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9"
Jul 07 13:21:32.837797 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:21:32.837776    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9\": container with ID starting with 2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9 not found: ID does not exist" containerID="2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9"
Jul 07 13:21:32.837852 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.837808    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9"} err="failed to get container status \"2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9\": rpc error: code = NotFound desc = could not find container \"2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9\": container with ID starting with 2052a3c0f60454b633d5afa2865b669f5f921c1aaf6463c93f84f94781fc58b9 not found: ID does not exist"
Jul 07 13:21:32.849851 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.849827    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"]
Jul 07 13:21:32.854163 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.854139    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0ebff7/rate-test-model-0ebff7-kserve-6f7b4d4879-pjhz6"]
Jul 07 13:21:32.894185 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.894147    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:32.894185 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.894181    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5btk\" (UniqueName: \"kubernetes.io/projected/a0beb99b-2a7c-4fbb-b35e-26c231085c03-kube-api-access-h5btk\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:32.894185 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.894192    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:32.894398 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:32.894201    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0beb99b-2a7c-4fbb-b35e-26c231085c03-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:33.617733 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.617686    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n" podUID="544779b9-d61f-4c1d-b429-e2e61742c107" containerName="main" containerID="cri-o://593d332c38205727aae5dcb56ef8f566736f51e8b84a6018fce8ba8a86b4f5f6" gracePeriod=45
Jul 07 13:21:33.836537 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.836496    2579 generic.go:356] "Generic (PLEG): container finished" podID="544779b9-d61f-4c1d-b429-e2e61742c107" containerID="593d332c38205727aae5dcb56ef8f566736f51e8b84a6018fce8ba8a86b4f5f6" exitCode=0
Jul 07 13:21:33.836719 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.836577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n" event={"ID":"544779b9-d61f-4c1d-b429-e2e61742c107","Type":"ContainerDied","Data":"593d332c38205727aae5dcb56ef8f566736f51e8b84a6018fce8ba8a86b4f5f6"}
Jul 07 13:21:33.860049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.860022    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:21:33.901822 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.901741    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-model-cache\") pod \"544779b9-d61f-4c1d-b429-e2e61742c107\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") "
Jul 07 13:21:33.901822 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.901795    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-tmp-dir\") pod \"544779b9-d61f-4c1d-b429-e2e61742c107\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") "
Jul 07 13:21:33.902049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.901834    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l542t\" (UniqueName: \"kubernetes.io/projected/544779b9-d61f-4c1d-b429-e2e61742c107-kube-api-access-l542t\") pod \"544779b9-d61f-4c1d-b429-e2e61742c107\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") "
Jul 07 13:21:33.902049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.901854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-dshm\") pod \"544779b9-d61f-4c1d-b429-e2e61742c107\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") "
Jul 07 13:21:33.902049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.901872    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-home\") pod \"544779b9-d61f-4c1d-b429-e2e61742c107\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") "
Jul 07 13:21:33.902049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.901899    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/544779b9-d61f-4c1d-b429-e2e61742c107-tls-certs\") pod \"544779b9-d61f-4c1d-b429-e2e61742c107\" (UID: \"544779b9-d61f-4c1d-b429-e2e61742c107\") "
Jul 07 13:21:33.902049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.902021    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-model-cache" (OuterVolumeSpecName: "model-cache") pod "544779b9-d61f-4c1d-b429-e2e61742c107" (UID: "544779b9-d61f-4c1d-b429-e2e61742c107"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:33.902049 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.902026    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "544779b9-d61f-4c1d-b429-e2e61742c107" (UID: "544779b9-d61f-4c1d-b429-e2e61742c107"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:33.902340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.902140    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-home" (OuterVolumeSpecName: "home") pod "544779b9-d61f-4c1d-b429-e2e61742c107" (UID: "544779b9-d61f-4c1d-b429-e2e61742c107"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:33.902340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.902243    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-home\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:33.902340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.902264    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-model-cache\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:33.902340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.902277    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-tmp-dir\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:33.903981 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.903958    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-dshm" (OuterVolumeSpecName: "dshm") pod "544779b9-d61f-4c1d-b429-e2e61742c107" (UID: "544779b9-d61f-4c1d-b429-e2e61742c107"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 13:21:33.904083 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.904050    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544779b9-d61f-4c1d-b429-e2e61742c107-kube-api-access-l542t" (OuterVolumeSpecName: "kube-api-access-l542t") pod "544779b9-d61f-4c1d-b429-e2e61742c107" (UID: "544779b9-d61f-4c1d-b429-e2e61742c107"). InnerVolumeSpecName "kube-api-access-l542t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:21:33.904140 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:33.904087    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/544779b9-d61f-4c1d-b429-e2e61742c107-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "544779b9-d61f-4c1d-b429-e2e61742c107" (UID: "544779b9-d61f-4c1d-b429-e2e61742c107"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:21:34.003312 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.003276    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l542t\" (UniqueName: \"kubernetes.io/projected/544779b9-d61f-4c1d-b429-e2e61742c107-kube-api-access-l542t\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:34.003312 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.003307    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/544779b9-d61f-4c1d-b429-e2e61742c107-dshm\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:34.003312 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.003319    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/544779b9-d61f-4c1d-b429-e2e61742c107-tls-certs\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:21:34.517798 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.517755    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0beb99b-2a7c-4fbb-b35e-26c231085c03" path="/var/lib/kubelet/pods/a0beb99b-2a7c-4fbb-b35e-26c231085c03/volumes"
Jul 07 13:21:34.842697 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.842665    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"
Jul 07 13:21:34.842697 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.842684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n" event={"ID":"544779b9-d61f-4c1d-b429-e2e61742c107","Type":"ContainerDied","Data":"43fb19faf4c7e7f559b42dfae2eafe1b827cadc9c6c5cc1555ae8b9fd9d920f8"}
Jul 07 13:21:34.843208 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.842729    2579 scope.go:117] "RemoveContainer" containerID="593d332c38205727aae5dcb56ef8f566736f51e8b84a6018fce8ba8a86b4f5f6"
Jul 07 13:21:34.859701 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.859673    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"]
Jul 07 13:21:34.863079 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:34.863055    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e80c89/rate-test-model-e80c89-kserve-964f76455-7dl6n"]
Jul 07 13:21:36.518658 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:21:36.518623    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="544779b9-d61f-4c1d-b429-e2e61742c107" path="/var/lib/kubelet/pods/544779b9-d61f-4c1d-b429-e2e61742c107/volumes"
Jul 07 13:22:03.989748 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:03.989713    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"]
Jul 07 13:22:03.990315 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:03.989941    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" podUID="10d1533b-0d7e-4a23-826b-a07d6c9881cf" containerName="maas-api" containerID="cri-o://415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367" gracePeriod=30
Jul 07 13:22:04.245327 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.245264    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:22:04.353134 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.353099    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d1533b-0d7e-4a23-826b-a07d6c9881cf-openshift-service-ca\") pod \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") "
Jul 07 13:22:04.353303 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.353145    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls\") pod \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") "
Jul 07 13:22:04.353303 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.353175    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9mwj\" (UniqueName: \"kubernetes.io/projected/10d1533b-0d7e-4a23-826b-a07d6c9881cf-kube-api-access-l9mwj\") pod \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\" (UID: \"10d1533b-0d7e-4a23-826b-a07d6c9881cf\") "
Jul 07 13:22:04.353685 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.353651    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10d1533b-0d7e-4a23-826b-a07d6c9881cf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10d1533b-0d7e-4a23-826b-a07d6c9881cf" (UID: "10d1533b-0d7e-4a23-826b-a07d6c9881cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 13:22:04.355371 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.355341    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10d1533b-0d7e-4a23-826b-a07d6c9881cf" (UID: "10d1533b-0d7e-4a23-826b-a07d6c9881cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 13:22:04.355371 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.355348    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d1533b-0d7e-4a23-826b-a07d6c9881cf-kube-api-access-l9mwj" (OuterVolumeSpecName: "kube-api-access-l9mwj") pod "10d1533b-0d7e-4a23-826b-a07d6c9881cf" (UID: "10d1533b-0d7e-4a23-826b-a07d6c9881cf"). InnerVolumeSpecName "kube-api-access-l9mwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 13:22:04.454260 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.454219    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10d1533b-0d7e-4a23-826b-a07d6c9881cf-maas-api-tls\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:22:04.454260 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.454253    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9mwj\" (UniqueName: \"kubernetes.io/projected/10d1533b-0d7e-4a23-826b-a07d6c9881cf-kube-api-access-l9mwj\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:22:04.454260 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.454265    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10d1533b-0d7e-4a23-826b-a07d6c9881cf-openshift-service-ca\") on node \"ip-10-0-132-141.ec2.internal\" DevicePath \"\""
Jul 07 13:22:04.992891 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.992851    2579 generic.go:356] "Generic (PLEG): container finished" podID="10d1533b-0d7e-4a23-826b-a07d6c9881cf" containerID="415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367" exitCode=0
Jul 07 13:22:04.993370 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.992901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"
Jul 07 13:22:04.993370 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.992935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" event={"ID":"10d1533b-0d7e-4a23-826b-a07d6c9881cf","Type":"ContainerDied","Data":"415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367"}
Jul 07 13:22:04.993370 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.992982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7" event={"ID":"10d1533b-0d7e-4a23-826b-a07d6c9881cf","Type":"ContainerDied","Data":"687d7c310378973a19c545da2b0d6a1ed2ce21c4a1b19055e658f32fc1cde497"}
Jul 07 13:22:04.993370 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:04.993005    2579 scope.go:117] "RemoveContainer" containerID="415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367"
Jul 07 13:22:05.001649 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:05.001630    2579 scope.go:117] "RemoveContainer" containerID="415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367"
Jul 07 13:22:05.001923 ip-10-0-132-141 kubenswrapper[2579]: E0707 13:22:05.001905    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367\": container with ID starting with 415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367 not found: ID does not exist" containerID="415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367"
Jul 07 13:22:05.001974 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:05.001933    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367"} err="failed to get container status \"415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367\": rpc error: code = NotFound desc = could not find container \"415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367\": container with ID starting with 415cbcd89b38e46e6489c97d03152e9364c9595ff49ff497b2f542d688624367 not found: ID does not exist"
Jul 07 13:22:05.023066 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:05.023033    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"]
Jul 07 13:22:05.027718 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:05.027686    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e80c89-7c96fd95dd-z76h7"]
Jul 07 13:22:06.517116 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:06.517083    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10d1533b-0d7e-4a23-826b-a07d6c9881cf" path="/var/lib/kubelet/pods/10d1533b-0d7e-4a23-826b-a07d6c9881cf/volumes"
Jul 07 13:22:07.381030 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:07.380992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sqzl2"]
Jul 07 13:22:11.210669 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:11.210634    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b68756c4c-4d4zl_1af2a399-201c-4105-a1f0-a38ef29d26f4/authorino/0.log"
Jul 07 13:22:16.666387 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:16.666357    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-d84kw_4aac462d-57f4-445e-ab64-daf261410896/copy-manifests/0.log"
Jul 07 13:22:16.674340 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:16.674318    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-d84kw_4aac462d-57f4-445e-ab64-daf261410896/manager/0.log"
Jul 07 13:22:17.787185 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:17.787151    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd_ce07c240-0c91-455e-9fed-3174530ab721/util/0.log"
Jul 07 13:22:17.800428 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:17.800407    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd_ce07c240-0c91-455e-9fed-3174530ab721/pull/0.log"
Jul 07 13:22:17.814804 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:17.814778    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd_ce07c240-0c91-455e-9fed-3174530ab721/extract/0.log"
Jul 07 13:22:17.939465 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:17.939414    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x_3d16359c-14b0-49af-a484-16d703780bd2/pull/0.log"
Jul 07 13:22:17.946597 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:17.946576    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x_3d16359c-14b0-49af-a484-16d703780bd2/extract/0.log"
Jul 07 13:22:17.953226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:17.953211    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x_3d16359c-14b0-49af-a484-16d703780bd2/util/0.log"
Jul 07 13:22:18.077410 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.077389    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp_e45cc3af-0bb3-4eaa-b798-75e4ff60c012/util/0.log"
Jul 07 13:22:18.084249 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.084224    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp_e45cc3af-0bb3-4eaa-b798-75e4ff60c012/pull/0.log"
Jul 07 13:22:18.091867 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.091845    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp_e45cc3af-0bb3-4eaa-b798-75e4ff60c012/extract/0.log"
Jul 07 13:22:18.210822 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.210791    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh_26b6855d-2915-4943-a742-35629e91fe52/util/0.log"
Jul 07 13:22:18.218824 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.218794    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh_26b6855d-2915-4943-a742-35629e91fe52/pull/0.log"
Jul 07 13:22:18.226129 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.226107    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh_26b6855d-2915-4943-a742-35629e91fe52/extract/0.log"
Jul 07 13:22:18.359757 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.359680    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b68756c4c-4d4zl_1af2a399-201c-4105-a1f0-a38ef29d26f4/authorino/0.log"
Jul 07 13:22:18.739382 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:18.739295    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-92rtf_52abd8ae-443c-4c01-9918-baa16a0e14a7/kuadrant-console-plugin/0.log"
Jul 07 13:22:19.120406 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:19.120374    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sqzl2_2e732c66-ea49-4b2e-a0d0-4a2e44e52d55/limitador/0.log"
Jul 07 13:22:19.910620 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:19.910572    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-4ld4x_69959d63-f09d-448c-811c-9c948a867a39/kube-auth-proxy/0.log"
Jul 07 13:22:21.517633 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:21.517605    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-gdfrw_2fb4a324-8007-4e86-9c6f-07e9f06a1ee0/main/0.log"
Jul 07 13:22:25.570004 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.569962    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gwgv6/must-gather-qt9pm"]
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d1533b-0d7e-4a23-826b-a07d6c9881cf" containerName="maas-api"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570399    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d1533b-0d7e-4a23-826b-a07d6c9881cf" containerName="maas-api"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570424    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0beb99b-2a7c-4fbb-b35e-26c231085c03" containerName="main"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570432    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0beb99b-2a7c-4fbb-b35e-26c231085c03" containerName="main"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570445    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65652e62-199e-498f-be20-74ab69eb773a" containerName="maas-api"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570452    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="65652e62-199e-498f-be20-74ab69eb773a" containerName="maas-api"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570462    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544779b9-d61f-4c1d-b429-e2e61742c107" containerName="main"
Jul 07 13:22:25.570507 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570469    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="544779b9-d61f-4c1d-b429-e2e61742c107" containerName="main"
Jul 07 13:22:25.570962 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570566    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="65652e62-199e-498f-be20-74ab69eb773a" containerName="maas-api"
Jul 07 13:22:25.570962 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570579    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0beb99b-2a7c-4fbb-b35e-26c231085c03" containerName="main"
Jul 07 13:22:25.570962 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570591    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d1533b-0d7e-4a23-826b-a07d6c9881cf" containerName="maas-api"
Jul 07 13:22:25.570962 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.570600    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="544779b9-d61f-4c1d-b429-e2e61742c107" containerName="main"
Jul 07 13:22:25.573701 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.573680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.575698 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.575673    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gwgv6\"/\"openshift-service-ca.crt\""
Jul 07 13:22:25.575829 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.575762    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gwgv6\"/\"kube-root-ca.crt\""
Jul 07 13:22:25.576189 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.576175    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gwgv6\"/\"default-dockercfg-65jtm\""
Jul 07 13:22:25.580256 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.580229    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/must-gather-qt9pm"]
Jul 07 13:22:25.742735 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.742702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jv46\" (UniqueName: \"kubernetes.io/projected/bf35f5e1-e1dc-4a3c-8d14-554e96788ad9-kube-api-access-4jv46\") pod \"must-gather-qt9pm\" (UID: \"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9\") " pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.742926 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.742777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bf35f5e1-e1dc-4a3c-8d14-554e96788ad9-must-gather-output\") pod \"must-gather-qt9pm\" (UID: \"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9\") " pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.843958 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.843853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bf35f5e1-e1dc-4a3c-8d14-554e96788ad9-must-gather-output\") pod \"must-gather-qt9pm\" (UID: \"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9\") " pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.843958 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.843918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jv46\" (UniqueName: \"kubernetes.io/projected/bf35f5e1-e1dc-4a3c-8d14-554e96788ad9-kube-api-access-4jv46\") pod \"must-gather-qt9pm\" (UID: \"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9\") " pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.844260 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.844230    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bf35f5e1-e1dc-4a3c-8d14-554e96788ad9-must-gather-output\") pod \"must-gather-qt9pm\" (UID: \"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9\") " pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.851226 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.851203    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jv46\" (UniqueName: \"kubernetes.io/projected/bf35f5e1-e1dc-4a3c-8d14-554e96788ad9-kube-api-access-4jv46\") pod \"must-gather-qt9pm\" (UID: \"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9\") " pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:25.884181 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:25.884137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/must-gather-qt9pm"
Jul 07 13:22:26.017111 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:26.017089    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/must-gather-qt9pm"]
Jul 07 13:22:26.019145 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:22:26.019120    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf35f5e1_e1dc_4a3c_8d14_554e96788ad9.slice/crio-a67a46c66605cf53e689cd92fe3a335827c810910f3ff1fbf9dd8d0009cd7209 WatchSource:0}: Error finding container a67a46c66605cf53e689cd92fe3a335827c810910f3ff1fbf9dd8d0009cd7209: Status 404 returned error can't find the container with id a67a46c66605cf53e689cd92fe3a335827c810910f3ff1fbf9dd8d0009cd7209
Jul 07 13:22:26.021074 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:26.021057    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 13:22:26.095972 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:26.095881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/must-gather-qt9pm" event={"ID":"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9","Type":"ContainerStarted","Data":"a67a46c66605cf53e689cd92fe3a335827c810910f3ff1fbf9dd8d0009cd7209"}
Jul 07 13:22:28.107200 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:28.107161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/must-gather-qt9pm" event={"ID":"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9","Type":"ContainerStarted","Data":"0fbfd4f8eee198493a1fa5f167f394ea76982f9ba9fb92d46d4d270ad745d831"}
Jul 07 13:22:28.107632 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:28.107213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/must-gather-qt9pm" event={"ID":"bf35f5e1-e1dc-4a3c-8d14-554e96788ad9","Type":"ContainerStarted","Data":"4280e3619ad7ce44a3ca6ef37db775aa6092f936958f567dc07ce2541656c9ac"}
Jul 07 13:22:28.128667 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:28.128598    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gwgv6/must-gather-qt9pm" podStartSLOduration=1.754968094 podStartE2EDuration="3.128575022s" totalImagesPullingTime="1.373606928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:28.125906004 +0000 UTC m=+2416.126772905" watchObservedRunningTime="2026-07-07 13:22:28.128575022 +0000 UTC m=+2416.129441943"
Jul 07 13:22:29.169557 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:29.169502    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9ngf2_2a0476f5-c151-4bec-9b97-adc1b69ea8b5/global-pull-secret-syncer/0.log"
Jul 07 13:22:29.480054 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:29.479957    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6zwxt_ed74092e-b808-4be1-8b12-c57dae291412/konnectivity-agent/0.log"
Jul 07 13:22:29.608688 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:29.608659    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-141.ec2.internal_fe845dad01dc41ef76e1b98e2c39b5a0/haproxy/0.log"
Jul 07 13:22:33.326467 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.326384    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd_ce07c240-0c91-455e-9fed-3174530ab721/extract/0.log"
Jul 07 13:22:33.365392 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.365354    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd_ce07c240-0c91-455e-9fed-3174530ab721/util/0.log"
Jul 07 13:22:33.398024 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.397972    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xjgrd_ce07c240-0c91-455e-9fed-3174530ab721/pull/0.log"
Jul 07 13:22:33.443705 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.443670    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x_3d16359c-14b0-49af-a484-16d703780bd2/extract/0.log"
Jul 07 13:22:33.474717 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.474690    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x_3d16359c-14b0-49af-a484-16d703780bd2/util/0.log"
Jul 07 13:22:33.500346 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.500306    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e098q4x_3d16359c-14b0-49af-a484-16d703780bd2/pull/0.log"
Jul 07 13:22:33.531451 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.531427    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp_e45cc3af-0bb3-4eaa-b798-75e4ff60c012/extract/0.log"
Jul 07 13:22:33.554394 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.554368    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp_e45cc3af-0bb3-4eaa-b798-75e4ff60c012/util/0.log"
Jul 07 13:22:33.611150 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.611054    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tllcp_e45cc3af-0bb3-4eaa-b798-75e4ff60c012/pull/0.log"
Jul 07 13:22:33.647458 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.647391    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh_26b6855d-2915-4943-a742-35629e91fe52/extract/0.log"
Jul 07 13:22:33.671643 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.671611    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh_26b6855d-2915-4943-a742-35629e91fe52/util/0.log"
Jul 07 13:22:33.700079 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:33.700047    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rlfsh_26b6855d-2915-4943-a742-35629e91fe52/pull/0.log"
Jul 07 13:22:34.085726 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:34.085503    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b68756c4c-4d4zl_1af2a399-201c-4105-a1f0-a38ef29d26f4/authorino/0.log"
Jul 07 13:22:34.172121 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:34.172089    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-92rtf_52abd8ae-443c-4c01-9918-baa16a0e14a7/kuadrant-console-plugin/0.log"
Jul 07 13:22:34.394007 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:34.393936    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sqzl2_2e732c66-ea49-4b2e-a0d0-4a2e44e52d55/limitador/0.log"
Jul 07 13:22:36.567372 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:36.567299    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bpsxz_cddd83b3-c896-4ded-81b6-7ea26d8bbf2f/node-exporter/0.log"
Jul 07 13:22:36.598355 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:36.598316    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bpsxz_cddd83b3-c896-4ded-81b6-7ea26d8bbf2f/kube-rbac-proxy/0.log"
Jul 07 13:22:36.632495 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:36.632464    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bpsxz_cddd83b3-c896-4ded-81b6-7ea26d8bbf2f/init-textfile/0.log"
Jul 07 13:22:37.112274 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:37.112230    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-m788c_f85b4193-e750-4055-be6f-307944a5d7c6/prometheus-operator-admission-webhook/0.log"
Jul 07 13:22:39.243802 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.243760    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"]
Jul 07 13:22:39.260271 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.260238    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"]
Jul 07 13:22:39.260448 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.260360    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.377781 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.377732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-podres\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.378143 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.378119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-proc\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.378307 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.378291    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-sys\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.378447 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.378430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8rj\" (UniqueName: \"kubernetes.io/projected/846a6e96-043b-4084-8c8e-05bfb046bba3-kube-api-access-nl8rj\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.378598 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.378584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-lib-modules\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479501 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-proc\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479712 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-sys\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479712 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8rj\" (UniqueName: \"kubernetes.io/projected/846a6e96-043b-4084-8c8e-05bfb046bba3-kube-api-access-nl8rj\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479712 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-lib-modules\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479712 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479614    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-proc\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479712 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479687    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-sys\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479732    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-lib-modules\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-podres\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.479966 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.479861    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/846a6e96-043b-4084-8c8e-05bfb046bba3-podres\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.488177 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.488149    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8rj\" (UniqueName: \"kubernetes.io/projected/846a6e96-043b-4084-8c8e-05bfb046bba3-kube-api-access-nl8rj\") pod \"perf-node-gather-daemonset-4cp2j\" (UID: \"846a6e96-043b-4084-8c8e-05bfb046bba3\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.572067 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.572035    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:39.753791 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.753439    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"]
Jul 07 13:22:39.756652 ip-10-0-132-141 kubenswrapper[2579]: W0707 13:22:39.756624    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod846a6e96_043b_4084_8c8e_05bfb046bba3.slice/crio-d7cb574abacf1b896bd612d0fcf4211c27583d78e6f20722b8724ec627087e30 WatchSource:0}: Error finding container d7cb574abacf1b896bd612d0fcf4211c27583d78e6f20722b8724ec627087e30: Status 404 returned error can't find the container with id d7cb574abacf1b896bd612d0fcf4211c27583d78e6f20722b8724ec627087e30
Jul 07 13:22:39.893057 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:39.893018    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-bcfsc_b95f988e-bbee-42d0-b2df-3a702689b803/download-server/0.log"
Jul 07 13:22:40.193241 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:40.193149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j" event={"ID":"846a6e96-043b-4084-8c8e-05bfb046bba3","Type":"ContainerStarted","Data":"85eb5b74d9f9005a8668f08f89cda4550b4007b67e4a2c34562e41e0ee24be57"}
Jul 07 13:22:40.193241 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:40.193208    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j" event={"ID":"846a6e96-043b-4084-8c8e-05bfb046bba3","Type":"ContainerStarted","Data":"d7cb574abacf1b896bd612d0fcf4211c27583d78e6f20722b8724ec627087e30"}
Jul 07 13:22:40.193483 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:40.193260    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:41.730458 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:41.730435    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c5h8s_055e0d3d-3f8f-4d6a-acb3-7372795eda3f/dns/0.log"
Jul 07 13:22:41.751596 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:41.751573    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c5h8s_055e0d3d-3f8f-4d6a-acb3-7372795eda3f/kube-rbac-proxy/0.log"
Jul 07 13:22:41.851024 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:41.850999    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cs8fp_aebd87aa-c237-4a76-8d8d-80c8072cd0da/dns-node-resolver/0.log"
Jul 07 13:22:42.416616 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:42.416577    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8949fc898-wcj5r_b9184802-2a49-4003-afda-e994a598babc/registry/0.log"
Jul 07 13:22:42.439905 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:42.439875    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-272qj_f4ac942c-b28d-40da-b650-0451e463eb97/node-ca/0.log"
Jul 07 13:22:43.696353 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:43.696327    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-4ld4x_69959d63-f09d-448c-811c-9c948a867a39/kube-auth-proxy/0.log"
Jul 07 13:22:44.594763 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:44.594735    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gprxp_d94a46e3-a622-44a6-9fb2-d4bdaf2c0e2f/serve-healthcheck-canary/0.log"
Jul 07 13:22:45.170917 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:45.170881    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hmpsl_31eee505-35d2-4037-8cf6-4a4263b4ee78/kube-rbac-proxy/0.log"
Jul 07 13:22:45.197171 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:45.197140    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hmpsl_31eee505-35d2-4037-8cf6-4a4263b4ee78/exporter/0.log"
Jul 07 13:22:45.225457 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:45.225426    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hmpsl_31eee505-35d2-4037-8cf6-4a4263b4ee78/extractor/0.log"
Jul 07 13:22:46.212291 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:46.212265    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j"
Jul 07 13:22:46.229823 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:46.229768    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-4cp2j" podStartSLOduration=7.229753118 podStartE2EDuration="7.229753118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:40.215690682 +0000 UTC m=+2428.216557601" watchObservedRunningTime="2026-07-07 13:22:46.229753118 +0000 UTC m=+2434.230620034"
Jul 07 13:22:47.985804 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:47.985769    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-d84kw_4aac462d-57f4-445e-ab64-daf261410896/manager/0.log"
Jul 07 13:22:48.006448 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:48.006412    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-d84kw_4aac462d-57f4-445e-ab64-daf261410896/copy-manifests/0.log"
Jul 07 13:22:49.577490 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:49.577455    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65fd447f77-pmgdw_81f6f2ce-2d2a-401b-9b25-d2ca9f3d4447/manager/0.log"
Jul 07 13:22:49.603878 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:49.603853    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gbblb_515510f9-cd07-4fa3-b3da-493612290a4a/openshift-lws-operator/0.log"
Jul 07 13:22:49.890097 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:49.890024    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-4ld4x_69959d63-f09d-448c-811c-9c948a867a39/kube-auth-proxy/0.log"
Jul 07 13:22:51.508464 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:51.508441    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74684786d9-4ld4x_69959d63-f09d-448c-811c-9c948a867a39/kube-auth-proxy/0.log"
Jul 07 13:22:55.532442 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:55.532413    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vwj5c_63227d1c-ede5-4d30-9620-87df207701ad/migrator/0.log"
Jul 07 13:22:55.590506 ip-10-0-132-141 kubenswrapper[2579]: I0707 13:22:55.590476    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vwj5c_63227d1c-ede5-4d30-9620-87df207701ad/graceful-termination/0.log"