Jul 06 23:56:07.476755 ip-10-0-141-123 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 23:56:07.910065 ip-10-0-141-123 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:07.910065 ip-10-0-141-123 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 23:56:07.910065 ip-10-0-141-123 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:07.910065 ip-10-0-141-123 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 23:56:07.910065 ip-10-0-141-123 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:07.912252 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.912154    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 23:56:07.917016 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917001    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:07.917016 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917016    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917020    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917023    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917025    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917028    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917031    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917034    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917040    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917043    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917046    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917049    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917052    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917054    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917057    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917059    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917062    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917064    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917067    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917069    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917072    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:07.917080 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917075    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917077    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917080    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917083    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917086    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917088    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917093    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917096    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917099    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917102    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917105    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917108    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917111    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917114    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917116    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917120    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917123    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917125    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917128    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:07.917565 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917130    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917133    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917135    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917138    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917140    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917143    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917146    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917148    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917151    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917153    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917156    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917158    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917161    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917163    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917166    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917169    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917172    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917174    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917177    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917179    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:07.918049 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917182    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917184    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917187    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917190    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917193    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917195    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917198    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917201    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917203    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917206    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917209    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917211    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917213    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917216    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917219    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917221    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917223    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917226    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917228    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917231    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:07.918586 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917234    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917236    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917239    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917241    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917244    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917246    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917250    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917254    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917256    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917259    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917261    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917264    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917266    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917269    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917271    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917274    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917277    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917281    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917284    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917968    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:07.919090 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917977    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917980    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917983    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917986    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917989    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917991    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917994    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917996    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.917999    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918002    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918004    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918008    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918011    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918014    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918016    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918019    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918022    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918024    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918027    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918030    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:07.919591 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918032    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918035    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918037    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918040    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918042    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918047    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918050    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918053    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918056    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918058    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918061    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918064    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918067    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918070    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918072    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918075    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918078    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918080    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918084    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918086    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:07.920117 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918089    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918091    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918094    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918096    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918100    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918102    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918105    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918107    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918110    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918112    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918115    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918118    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918121    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918123    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918126    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918128    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918131    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918133    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918136    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:07.920669 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918138    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918141    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918143    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918146    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918149    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918151    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918153    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918157    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918161    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918163    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918166    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918169    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918171    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918174    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918176    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918179    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918181    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918184    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918187    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:07.921146 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918190    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918193    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918195    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918198    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918201    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918204    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918206    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918210    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918213    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918216    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918219    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918222    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918225    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918227    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918230    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918233    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918235    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918238    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918240    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918243    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:07.921631 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918603    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918608    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918611    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918613    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918616    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918619    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918622    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918625    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918627    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918630    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918632    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918635    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918638    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918640    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918643    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918645    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918648    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918650    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918653    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918656    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918658    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:07.922387 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918661    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918663    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918666    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918668    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918671    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918673    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918676    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918678    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918681    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918683    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918686    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918688    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918691    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918693    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918696    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918698    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918701    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918703    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918706    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918709    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:07.922986 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918711    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918714    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918716    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918719    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918721    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918724    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918726    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918729    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918732    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918734    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918737    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918739    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918742    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918745    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918747    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918752    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918756    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918760    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918762    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:07.923492 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918765    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918768    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918770    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918774    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918777    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918780    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918783    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918785    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918788    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918791    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918793    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918796    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918798    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918801    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918804    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918806    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918809    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918811    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918814    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:07.924011 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918817    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918819    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918822    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918825    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918827    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918830    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918832    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918835    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918837    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918839    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918842    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918844    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918847    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918851    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918854    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918857    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918860    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918862    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918865    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:07.924554 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.918867    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918934    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918940    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918946    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918950    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918954    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918962    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918966    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918970    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918973    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918976    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918979    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918982    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918985    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918990    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918993    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918996    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.918998    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919002    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919006    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919009    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919012    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919015    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919018    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 23:56:07.925022 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919021    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919024    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919027    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919031    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919034    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919037    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919039    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919042    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919045    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919049    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919053    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919056    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919058    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919061    2564 flags.go:64] FLAG: --enable-server="true"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919065    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919069    2564 flags.go:64] FLAG: --event-burst="100"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919072    2564 flags.go:64] FLAG: --event-qps="50"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919074    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919077    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919081    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919084    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919087    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919091    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919094    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919098    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 23:56:07.925618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919101    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919104    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919107    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919109    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919112    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919115    2564 flags.go:64] FLAG: --feature-gates=""
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919119    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919121    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919124    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919127    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919130    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919133    2564 flags.go:64] FLAG: --help="false"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919136    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-141-123.ec2.internal"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919138    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919141    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919144    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919147    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919151    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919154    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919157    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919160    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919164    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919166    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919169    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 23:56:07.926284 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919172    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919175    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919178    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919181    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919183    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919187    2564 flags.go:64] FLAG: --lock-file=""
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919190    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919193    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919196    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919201    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919204    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919207    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919209    2564 flags.go:64] FLAG: --logging-format="text"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919212    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919215    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919218    2564 flags.go:64] FLAG: --manifest-url=""
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919221    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919224    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919227    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919231    2564 flags.go:64] FLAG: --max-pods="110"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919233    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919236    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919239    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919242    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919245    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 23:56:07.926900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919248    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919251    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919262    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919266    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919270    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919272    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919275    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919280    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919283    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919286    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919289    2564 flags.go:64] FLAG: --port="10250"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919291    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919296    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08cbc199151b9c2b5"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919299    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919302    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919305    2564 flags.go:64] FLAG: --register-node="true"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919308    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919311    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919314    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919317    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919319    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919323    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919326    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919329    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919332    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 23:56:07.927529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919335    2564 flags.go:64] FLAG: --runonce="false"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919338    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919341    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919344    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919346    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919349    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919352    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919355    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919358    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919360    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919364    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919367    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919371    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919373    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919376    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919379    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919384    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919387    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919389    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919394    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919398    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919401    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919403    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919406    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919409    2564 flags.go:64] FLAG: --v="2"
Jul 06 23:56:07.928317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919413    2564 flags.go:64] FLAG: --version="false"
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919416    2564 flags.go:64] FLAG: --vmodule=""
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919420    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.919423    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919547    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919553    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919557    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919560    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919562    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919565    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919567    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919570    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919573    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919575    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919578    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919581    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919583    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919586    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919588    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919593    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:07.929098 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919595    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919598    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919601    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919603    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919606    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919608    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919612    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919615    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919618    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919621    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919623    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919626    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919628    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919631    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919633    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919636    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919638    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919643    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919646    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919649    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:07.929633 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919652    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919654    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919658    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919660    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919663    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919666    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919668    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919671    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919673    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919676    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919678    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919683    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919685    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919688    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919690    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919693    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919695    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919698    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919701    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919704    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:07.930121 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919706    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919709    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919711    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919714    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919716    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919719    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919722    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919724    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919727    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919729    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919732    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919734    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919737    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919740    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919742    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919744    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919747    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919750    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919753    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919755    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:07.930650 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919758    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919760    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919763    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919767    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919770    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919772    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919775    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919777    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919780    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919783    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919787    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919789    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919793    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919796    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919799    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919802    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919804    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919807    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919809    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:07.931133 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919812    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919814    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919817    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.919819    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.920640    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.930407    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.930421    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.931317    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.931324    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931378    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931383    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931387    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931390    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931393    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:07.931694 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931396    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931398    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931401    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931403    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931406    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931409    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931411    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931414    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931418    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931422    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931425    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931428    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931430    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931433    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931435    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931438    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931440    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931443    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931445    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931448    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:07.932054 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931465    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931470    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931472    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931475    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931477    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931480    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931482    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931485    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931487    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931490    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931493    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931495    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931498    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931500    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931503    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931506    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931508    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931510    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931513    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:07.932567 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931515    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931518    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931521    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931523    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931526    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931528    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931530    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931533    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931537    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931541    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931545    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931549    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931553    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931556    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931559    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931561    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931564    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931567    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931569    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:07.933036 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931572    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931574    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931577    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931579    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931582    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931585    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931587    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931590    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931592    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931595    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931598    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931600    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931603    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931605    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931608    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931610    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931614    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931616    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931619    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931621    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:07.933512 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931624    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931626    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931629    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931631    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931634    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931636    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931639    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931642    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931644    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931647    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931649    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931652    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931654    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931657    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931660    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:07.933995 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931662    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.931667    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931778    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931783    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931786    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931789    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931792    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931794    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931797    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931800    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931802    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931805    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931808    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931811    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931814    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:07.934379 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931816    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931819    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931821    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931824    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931827    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931830    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931832    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931835    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931837    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931840    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931843    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931847    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931850    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931853    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931856    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931859    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931861    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931864    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931866    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:07.934767 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931869    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931871    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931874    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931876    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931880    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931882    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931885    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931887    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931890    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931892    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931895    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931897    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931900    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931903    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931906    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931908    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931911    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931914    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931917    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931919    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:07.935233 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931922    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931925    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931927    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931930    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931932    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931934    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931937    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931939    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931942    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931944    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931947    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931950    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931952    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931955    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931957    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931960    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931962    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931964    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931967    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931969    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:07.935905 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931972    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931974    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931977    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931979    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931982    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931984    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931987    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931990    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931992    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931996    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.931999    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932002    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932005    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932008    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932011    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932014    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932016    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932019    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932021    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:07.936410 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932024    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932026    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932029    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932031    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932033    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932036    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932038    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:07.932041    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.932045    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.932148    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 23:56:07.936888 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.935434    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 23:56:07.937227 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.937214    2564 server.go:1019] "Starting client certificate rotation"
Jul 06 23:56:07.937336 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.937319    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:07.937376 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.937351    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:07.940779 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.940765    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:07.941888 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.941871    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:07.952732 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.952718    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 06 23:56:07.955630 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.955614    2564 log.go:25] "Validated CRI v1 image API"
Jul 06 23:56:07.956525 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.956508    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 23:56:07.959110 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.959089    2564 fs.go:135] Filesystem UUIDs: map[20dd1bff-1c58-4051-8c5d-3447883a8a3e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a088bc2b-bb65-4c33-a279-d5f7a9b5da9b:/dev/nvme0n1p4]
Jul 06 23:56:07.959182 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.959109    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 23:56:07.965663 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.965545    2564 manager.go:216] Machine: {Timestamp:2026-07-06 23:56:07.963112501 +0000 UTC m=+0.376126401 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3061206 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec284e8444e02ee7e8acbafdaf9014f8 SystemUUID:ec284e84-44e0-2ee7-e8ac-bafdaf9014f8 BootID:932e2bc4-6993-420e-bed5-dbe50d765353 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:97:a0:1b:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:97:a0:1b:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:7b:fe:07:1c:3e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 23:56:07.965663 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.965650    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 23:56:07.965818 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.965749    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 23:56:07.966905 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.966877    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:07.968630 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.968607    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 23:56:07.968762 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.968636    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 23:56:07.969513 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.969501    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 23:56:07.969564 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.969518    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 23:56:07.969564 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.969526    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:07.970399 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.970386    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:07.972333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.972322    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 23:56:07.976954 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.976943    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:07.977067 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977058    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 23:56:07.977099 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977088    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 23:56:07.977099 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977097    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 23:56:07.977153 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977112    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 23:56:07.977153 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977120    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 06 23:56:07.977153 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977128    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 23:56:07.977525 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977491    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:07.977525 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.977517    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:07.980129 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.980104    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 23:56:07.982532 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.982519    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 23:56:07.982591 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.982549    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 23:56:07.984622 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984603    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 23:56:07.984622 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984625    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984630    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984636    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984644    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984650    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984658    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984662    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984669    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984674    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984682    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 23:56:07.984738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.984686    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 23:56:07.986389 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.986377    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 23:56:07.986389 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.986389    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 23:56:07.989143 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:07.989120    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 23:56:07.989218 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:07.989121    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 23:56:07.989738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.989726    2564 server.go:1268] "Started kubelet"
Jul 06 23:56:07.989815 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.989793    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 23:56:07.989936 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.989904    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 23:56:07.990035 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.989951    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 23:56:07.990492 ip-10-0-141-123 systemd[1]: Started Kubernetes Kubelet.
Jul 06 23:56:07.992550 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.992523    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 23:56:07.993964 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:07.993938    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 23:56:07.994594 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:07.994580    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 23:56:08.000645 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.000625    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 23:56:08.001703 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.000568    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-123.ec2.internal.18bfd8a20dcac84a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-123.ec2.internal,UID:ip-10-0-141-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-123.ec2.internal,},FirstTimestamp:2026-07-06 23:56:07.989700682 +0000 UTC m=+0.402714581,LastTimestamp:2026-07-06 23:56:07.989700682 +0000 UTC m=+0.402714581,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-123.ec2.internal,}"
Jul 06 23:56:08.003566 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.003536    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 23:56:08.003566 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.003540    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:08.004146 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.004129    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.004511 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.004496    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 23:56:08.004585 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.004532    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 23:56:08.004585 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.004545    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 23:56:08.004683 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.004647    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 23:56:08.004683 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.004658    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 23:56:08.005182 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005167    2564 factory.go:55] Registering systemd factory
Jul 06 23:56:08.005252 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005206    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 06 23:56:08.005442 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005422    2564 factory.go:153] Registering CRI-O factory
Jul 06 23:56:08.005442 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005445    2564 factory.go:223] Registration of the crio container factory successfully
Jul 06 23:56:08.005563 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005509    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 23:56:08.005563 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005530    2564 factory.go:103] Registering Raw factory
Jul 06 23:56:08.005563 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.005547    2564 manager.go:1196] Started watching for new ooms in manager
Jul 06 23:56:08.006176 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.006158    2564 manager.go:349] Starting recovery of all containers
Jul 06 23:56:08.014184 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.014152    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 23:56:08.014792 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.014764    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 23:56:08.015953 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.015812    2564 manager.go:354] Recovery completed
Jul 06 23:56:08.019539 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.019515    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q88bx"
Jul 06 23:56:08.020057 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.020046    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.022751 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.022736    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.022813 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.022762    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.022813 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.022772    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.023288 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.023272    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 23:56:08.023288 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.023287    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 23:56:08.023429 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.023281    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.023429 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.023305    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:08.024018 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.023993    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.024018 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.024018    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.024117 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.024030    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.024681 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.024608    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-123.ec2.internal.18bfd8a20fc3147c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-123.ec2.internal,UID:ip-10-0-141-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-123.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-123.ec2.internal,},FirstTimestamp:2026-07-06 23:56:08.022750332 +0000 UTC m=+0.435764231,LastTimestamp:2026-07-06 23:56:08.022750332 +0000 UTC m=+0.435764231,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-123.ec2.internal,}"
Jul 06 23:56:08.025479 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.025466    2564 policy_none.go:49] "None policy: Start"
Jul 06 23:56:08.025530 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.025485    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 23:56:08.025530 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.025498    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 23:56:08.027505 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.027495    2564 policy_none.go:47] "Start"
Jul 06 23:56:08.027700 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.027685    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q88bx"
Jul 06 23:56:08.068021 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068006    2564 manager.go:337] "Starting Device Plugin manager"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.068045    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068056    2564 server.go:87] "Starting device plugin registration server"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068272    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068282    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068408    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068484    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.068491    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.069245    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 23:56:08.088529 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.069312    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.106934 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.106903    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 23:56:08.108107 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.108089    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 23:56:08.108194 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.108135    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 23:56:08.108194 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.108157    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 23:56:08.108316 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.108194    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 23:56:08.110608 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.110593    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:08.168994 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.168925    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.169799 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.169774    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.169902 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.169813    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.169902 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.169828    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.169902 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.169858    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.176893 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.176879    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.176955 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.176900    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-123.ec2.internal\": node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.199255 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.199234    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.208547 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.208526    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"]
Jul 06 23:56:08.208623 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.208599    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.209368 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.209355    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.209530 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.209386    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.209530 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.209400    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.211851 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.211837    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.211963 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.211950    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.212011 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.211976    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.212791 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.212776    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.212875 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.212779    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.212875 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.212824    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.212875 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.212835    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.212875 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.212800    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.212875 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.212873    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.215150 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.215135    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.215232 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.215165    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:08.215915 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.215893    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:08.215915 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.215915    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:08.216017 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.215926    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:08.229023 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.229001    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-123.ec2.internal\" not found" node="ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.232931 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.232916    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-123.ec2.internal\" not found" node="ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.300159 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.300125    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.306444 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.306425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7c519ecbb7e86b3494895d9cd797fde-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal\" (UID: \"b7c519ecbb7e86b3494895d9cd797fde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.306533 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.306469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7c519ecbb7e86b3494895d9cd797fde-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal\" (UID: \"b7c519ecbb7e86b3494895d9cd797fde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.306533 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.306489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bc8b346ff26bd2da637db45021bfde3d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-123.ec2.internal\" (UID: \"bc8b346ff26bd2da637db45021bfde3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.400757 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.400722    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.407089 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.407066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7c519ecbb7e86b3494895d9cd797fde-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal\" (UID: \"b7c519ecbb7e86b3494895d9cd797fde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.407159 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.407098    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bc8b346ff26bd2da637db45021bfde3d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-123.ec2.internal\" (UID: \"bc8b346ff26bd2da637db45021bfde3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.407159 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.407117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7c519ecbb7e86b3494895d9cd797fde-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal\" (UID: \"b7c519ecbb7e86b3494895d9cd797fde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.407250 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.407163    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7c519ecbb7e86b3494895d9cd797fde-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal\" (UID: \"b7c519ecbb7e86b3494895d9cd797fde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.407250 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.407183    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bc8b346ff26bd2da637db45021bfde3d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-123.ec2.internal\" (UID: \"bc8b346ff26bd2da637db45021bfde3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.407250 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.407167    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7c519ecbb7e86b3494895d9cd797fde-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal\" (UID: \"b7c519ecbb7e86b3494895d9cd797fde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.501570 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.501494    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.530887 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.530857    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.535818 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.535679    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:08.602735 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.602696    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.702848 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.702813    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.804871 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.804793    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.911324 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:08.911288    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:08.936686 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.936659    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 23:56:08.936841 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:08.936817    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:56:09.011957 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:09.011930    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:09.034144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.034106    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:51:08 +0000 UTC" deadline="2028-03-15 23:59:56.095369104 +0000 UTC"
Jul 06 23:56:09.034144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.034138    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14832h3m47.061234363s"
Jul 06 23:56:09.112427 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:09.112402    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:09.120738 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:09.120700    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c519ecbb7e86b3494895d9cd797fde.slice/crio-e63cca73efa2ecae678a0fa1d4d390110d7c937c4572406d48672ebd00a5379f WatchSource:0}: Error finding container e63cca73efa2ecae678a0fa1d4d390110d7c937c4572406d48672ebd00a5379f: Status 404 returned error can't find the container with id e63cca73efa2ecae678a0fa1d4d390110d7c937c4572406d48672ebd00a5379f
Jul 06 23:56:09.126907 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.126887    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:56:09.133450 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:09.133430    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8b346ff26bd2da637db45021bfde3d.slice/crio-5f8f55fbbdbd28d643c23471a059eb09e9274487be4ae25b3aa26bcfe8399d2c WatchSource:0}: Error finding container 5f8f55fbbdbd28d643c23471a059eb09e9274487be4ae25b3aa26bcfe8399d2c: Status 404 returned error can't find the container with id 5f8f55fbbdbd28d643c23471a059eb09e9274487be4ae25b3aa26bcfe8399d2c
Jul 06 23:56:09.213197 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:09.213157    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:09.314219 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:09.314179    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-123.ec2.internal\" not found"
Jul 06 23:56:09.330448 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.330423    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:09.405290 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.405202    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:09.423744 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.423721    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:09.425796 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.425782    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal"
Jul 06 23:56:09.437314 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.437293    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:09.453183 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.453164    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:09.501152 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.501117    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:09.979179 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.979148    2564 apiserver.go:52] "Watching apiserver"
Jul 06 23:56:09.991813 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.991782    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 23:56:09.992362 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.992324    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-5rgcf","kube-system/konnectivity-agent-qvk4s","kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs","openshift-image-registry/node-ca-wsjmn","openshift-multus/multus-additional-cni-plugins-wdflb","openshift-multus/network-metrics-daemon-wt5k2","openshift-network-operator/iptables-alerter-wfhzc","openshift-cluster-node-tuning-operator/tuned-gtrlq","openshift-dns/node-resolver-b95gn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal","openshift-multus/multus-fgtff","openshift-network-diagnostics/network-check-target-ccgjm","openshift-ovn-kubernetes/ovnkube-node-xb55m"]
Jul 06 23:56:09.995236 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.995202    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:09.995372 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:09.995293    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:09.997578 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:09.997554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:09.997697 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:09.997633    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:10.000178 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.000149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.002824 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.002797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.005824 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.005802    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:10.005933 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.005913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.007142 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.007119    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 23:56:10.007277 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.007258    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 23:56:10.007357 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.007258    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.008211 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.008173    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 23:56:10.012388 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.012363    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.012885 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.012551    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 23:56:10.013604 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.013580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014121    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014222    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014237    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014375    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gxhrf\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014615    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xchlh\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014636    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014667    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 23:56:10.014819 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014721    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.015268 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014861    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.015268 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.014941    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xcf5v\""
Jul 06 23:56:10.017123 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.017094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.017220 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.017176    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:10.018971 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.018949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.019601 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-node-log\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.019699 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-cni-netd\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.019699 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019643    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38b5994e-7085-4771-b116-8828bdf914ef-ovn-node-metrics-cert\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.019826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-system-cni-dir\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.019826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019733    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:10.019826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mk2p\" (UniqueName: \"kubernetes.io/projected/1df4ec18-0a1d-4039-a90d-e7afc5fca917-kube-api-access-4mk2p\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:10.019826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-log-socket\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.019826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-env-overrides\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019879    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nd47\" (UniqueName: \"kubernetes.io/projected/ef56e8b2-be96-4539-9a30-6bba9c69ddae-kube-api-access-2nd47\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019917    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-etc-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.019998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-run-netns\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-sys-fs\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.020075 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-slash\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020100    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-systemd\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-device-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02710aba-44b6-4d11-8cc3-3e0d2cb50934-host\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cnibin\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cni-binary-copy\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56jwh\" (UniqueName: \"kubernetes.io/projected/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-kube-api-access-56jwh\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020299    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-kubelet\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020324    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-ovnkube-script-lib\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-os-release\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-systemd-units\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020456    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7s4d\" (UniqueName: \"kubernetes.io/projected/38b5994e-7085-4771-b116-8828bdf914ef-kube-api-access-w7s4d\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-registration-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.020783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-etc-selinux\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dms\" (UniqueName: \"kubernetes.io/projected/02710aba-44b6-4d11-8cc3-3e0d2cb50934-kube-api-access-b8dms\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-cni-bin\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-ovnkube-config\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-socket-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020673    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02710aba-44b6-4d11-8cc3-3e0d2cb50934-serviceca\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-var-lib-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.021584 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.020757    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-ovn\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.021978 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.021896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.023073 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.023053    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 23:56:10.023174 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.023085    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.023174 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.023055    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 23:56:10.024963 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.024653    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.027691 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.027668    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96dd5\""
Jul 06 23:56:10.027803 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.027700    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 23:56:10.027803 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.027722    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.027803 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.027732    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.028029 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.028006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.028372 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.028352    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.028557 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.028519    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.028557 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.028538    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 23:56:10.028734 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.028676    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qt4c8\""
Jul 06 23:56:10.029419 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.029400    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.029726 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.029703    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:10.029814 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.029750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 23:56:10.029962 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.029712    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7jsw\""
Jul 06 23:56:10.029962 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.029949    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 23:56:10.030135 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.030117    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-thv5f\""
Jul 06 23:56:10.030202 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.030154    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vl48l\""
Jul 06 23:56:10.031553 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.031436    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 23:56:10.033748 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.033475    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rn9p4\""
Jul 06 23:56:10.036021 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.036001    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:10.042878 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.042855    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:10.077138 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.077114    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nnh5q"
Jul 06 23:56:10.083488 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.083462    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nnh5q"
Jul 06 23:56:10.105834 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.105806    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 23:56:10.115048 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.114981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal" event={"ID":"bc8b346ff26bd2da637db45021bfde3d","Type":"ContainerStarted","Data":"5f8f55fbbdbd28d643c23471a059eb09e9274487be4ae25b3aa26bcfe8399d2c"}
Jul 06 23:56:10.116176 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.116149    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal" event={"ID":"b7c519ecbb7e86b3494895d9cd797fde","Type":"ContainerStarted","Data":"e63cca73efa2ecae678a0fa1d4d390110d7c937c4572406d48672ebd00a5379f"}
Jul 06 23:56:10.121416 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-os-release\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.121416 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-run\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.121601 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mk2p\" (UniqueName: \"kubernetes.io/projected/1df4ec18-0a1d-4039-a90d-e7afc5fca917-kube-api-access-4mk2p\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:10.121601 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-log-socket\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.121601 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121510    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.121758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-env-overrides\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.121758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121633    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-log-socket\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.121758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121686    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-run-ovn-kubernetes\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.121758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-kubelet\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.121758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdrm7\" (UniqueName: \"kubernetes.io/projected/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-kube-api-access-gdrm7\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.121758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3fe339e-c4ea-43ec-b999-369382b7f87c-tmp\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.122047 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjbs\" (UniqueName: \"kubernetes.io/projected/d3fe339e-c4ea-43ec-b999-369382b7f87c-kube-api-access-xfjbs\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.122047 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-system-cni-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.122047 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-cni-multus\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.122047 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.121934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.122047 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-run-netns\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-cni-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-socket-dir-parent\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122109    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-run-netns\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysconfig\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122127    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122146    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysctl-d\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-sys\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-lib-modules\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-slash\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-systemd\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122281    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-slash\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-cni-bin\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122313    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-env-overrides\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-daemon-config\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.122333 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122334    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-systemd\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-var-lib-kubelet\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b879347-20d5-4742-b1e9-06e4d2ab7870-host-slash\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02710aba-44b6-4d11-8cc3-3e0d2cb50934-host\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cnibin\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56jwh\" (UniqueName: \"kubernetes.io/projected/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-kube-api-access-56jwh\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-registration-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-dbus\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122547    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cnibin\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-os-release\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122565    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02710aba-44b6-4d11-8cc3-3e0d2cb50934-host\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-cni-binary-copy\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122617    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-conf-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122626    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-os-release\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122635    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-registration-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-cni-bin\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-netns\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.123080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122689    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-cni-bin\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122691    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-modprobe-d\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-tuned\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9930d01b-c2fb-45eb-950c-c36e2bda9296-agent-certs\") pod \"konnectivity-agent-qvk4s\" (UID: \"9930d01b-c2fb-45eb-950c-c36e2bda9296\") " pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-ovn\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122910    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-ovn\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-node-log\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122955    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-node-log\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecb0f702-58cc-4ffd-b2df-f480b889d76d-hosts-file\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.122990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-cnibin\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-systemd\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123038    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-host\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-system-cni-dir\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123129    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-system-cni-dir\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nd47\" (UniqueName: \"kubernetes.io/projected/ef56e8b2-be96-4539-9a30-6bba9c69ddae-kube-api-access-2nd47\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.123942 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-etc-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.123229    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-multus-certs\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-sys-fs\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.123359    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:10.623332311 +0000 UTC m=+3.036346223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123394    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-hostroot\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123406    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-sys-fs\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123443    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-etc-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123473    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-device-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123520    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-run-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123569    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-device-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.124826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h9wc\" (UniqueName: \"kubernetes.io/projected/7b879347-20d5-4742-b1e9-06e4d2ab7870-kube-api-access-2h9wc\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cni-binary-copy\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-kubelet\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-ovnkube-script-lib\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123741    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-kubelet\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysctl-conf\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9930d01b-c2fb-45eb-950c-c36e2bda9296-konnectivity-ca\") pod \"konnectivity-agent-qvk4s\" (UID: \"9930d01b-c2fb-45eb-950c-c36e2bda9296\") " pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-systemd-units\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123863    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7s4d\" (UniqueName: \"kubernetes.io/projected/38b5994e-7085-4771-b116-8828bdf914ef-kube-api-access-w7s4d\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-etc-selinux\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecb0f702-58cc-4ffd-b2df-f480b889d76d-tmp-dir\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123972    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.123972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j78fw\" (UniqueName: \"kubernetes.io/projected/ecb0f702-58cc-4ffd-b2df-f480b889d76d-kube-api-access-j78fw\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124010    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-k8s-cni-cncf-io\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.125515 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124042    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-etc-kubernetes\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124057    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-etc-selinux\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-kubernetes\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124132    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-systemd-units\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124172    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dms\" (UniqueName: \"kubernetes.io/projected/02710aba-44b6-4d11-8cc3-3e0d2cb50934-kube-api-access-b8dms\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124245    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-ovnkube-config\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-socket-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124275    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-ovnkube-script-lib\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-kubelet-config\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02710aba-44b6-4d11-8cc3-3e0d2cb50934-serviceca\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-var-lib-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-cni-netd\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38b5994e-7085-4771-b116-8828bdf914ef-ovn-node-metrics-cert\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7b879347-20d5-4742-b1e9-06e4d2ab7870-iptables-alerter-script\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124630    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-cni-binary-copy\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124637    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ef56e8b2-be96-4539-9a30-6bba9c69ddae-socket-dir\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.126141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124685    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-var-lib-openvswitch\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126797 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124703    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38b5994e-7085-4771-b116-8828bdf914ef-host-cni-netd\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126797 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.124837    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38b5994e-7085-4771-b116-8828bdf914ef-ovnkube-config\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.126797 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.125019    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 23:56:10.126797 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.125054    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02710aba-44b6-4d11-8cc3-3e0d2cb50934-serviceca\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.128972 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.128946    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38b5994e-7085-4771-b116-8828bdf914ef-ovn-node-metrics-cert\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.131268 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.131244    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:10.131378 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.131273    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:10.131378 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.131295    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:10.131378 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.131353    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:10.631335399 +0000 UTC m=+3.044349318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:10.133895 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.133812    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mk2p\" (UniqueName: \"kubernetes.io/projected/1df4ec18-0a1d-4039-a90d-e7afc5fca917-kube-api-access-4mk2p\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:10.134186 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.134163    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nd47\" (UniqueName: \"kubernetes.io/projected/ef56e8b2-be96-4539-9a30-6bba9c69ddae-kube-api-access-2nd47\") pod \"aws-ebs-csi-driver-node-lxfxs\" (UID: \"ef56e8b2-be96-4539-9a30-6bba9c69ddae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.134273 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.134179    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56jwh\" (UniqueName: \"kubernetes.io/projected/0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073-kube-api-access-56jwh\") pod \"multus-additional-cni-plugins-wdflb\" (UID: \"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073\") " pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.134273 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.134267    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7s4d\" (UniqueName: \"kubernetes.io/projected/38b5994e-7085-4771-b116-8828bdf914ef-kube-api-access-w7s4d\") pod \"ovnkube-node-xb55m\" (UID: \"38b5994e-7085-4771-b116-8828bdf914ef\") " pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.134367 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.134346    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dms\" (UniqueName: \"kubernetes.io/projected/02710aba-44b6-4d11-8cc3-3e0d2cb50934-kube-api-access-b8dms\") pod \"node-ca-wsjmn\" (UID: \"02710aba-44b6-4d11-8cc3-3e0d2cb50934\") " pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.225845 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7b879347-20d5-4742-b1e9-06e4d2ab7870-iptables-alerter-script\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-os-release\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-run\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-kubelet\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdrm7\" (UniqueName: \"kubernetes.io/projected/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-kube-api-access-gdrm7\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3fe339e-c4ea-43ec-b999-369382b7f87c-tmp\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjbs\" (UniqueName: \"kubernetes.io/projected/d3fe339e-c4ea-43ec-b999-369382b7f87c-kube-api-access-xfjbs\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225988    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-run\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-system-cni-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226040    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-kubelet\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-cni-multus\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226076    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-system-cni-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-cni-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-socket-dir-parent\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysconfig\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysctl-d\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-sys\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-lib-modules\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-cni-bin\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226278    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-socket-dir-parent\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.225986    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-os-release\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-daemon-config\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-var-lib-kubelet\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226341    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-cni-multus\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b879347-20d5-4742-b1e9-06e4d2ab7870-host-slash\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226422    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b879347-20d5-4742-b1e9-06e4d2ab7870-host-slash\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226423    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-var-lib-cni-bin\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.226555 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226468    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-sys\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226497    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-var-lib-kubelet\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226515    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-cni-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226544    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysconfig\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-dbus\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226578    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7b879347-20d5-4742-b1e9-06e4d2ab7870-iptables-alerter-script\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-cni-binary-copy\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226610    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-lib-modules\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-conf-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226657    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysctl-d\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226653    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-dbus\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226685    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-conf-dir\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-netns\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-modprobe-d\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226734    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-netns\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-tuned\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226839    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-modprobe-d\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9930d01b-c2fb-45eb-950c-c36e2bda9296-agent-certs\") pod \"konnectivity-agent-qvk4s\" (UID: \"9930d01b-c2fb-45eb-950c-c36e2bda9296\") " pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.227451 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecb0f702-58cc-4ffd-b2df-f480b889d76d-hosts-file\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-cnibin\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226879    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-multus-daemon-config\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.226993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-systemd\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-host\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227054    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-cni-binary-copy\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227069    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-cnibin\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227060    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecb0f702-58cc-4ffd-b2df-f480b889d76d-hosts-file\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-multus-certs\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227108    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-systemd\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227128    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-host\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-hostroot\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227172    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-hostroot\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227130    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-multus-certs\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h9wc\" (UniqueName: \"kubernetes.io/projected/7b879347-20d5-4742-b1e9-06e4d2ab7870-kube-api-access-2h9wc\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysctl-conf\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9930d01b-c2fb-45eb-950c-c36e2bda9296-konnectivity-ca\") pod \"konnectivity-agent-qvk4s\" (UID: \"9930d01b-c2fb-45eb-950c-c36e2bda9296\") " pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.228330 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.227271    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.227360    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:10.72734111 +0000 UTC m=+3.140355006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227397    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-sysctl-conf\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecb0f702-58cc-4ffd-b2df-f480b889d76d-tmp-dir\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227569    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecb0f702-58cc-4ffd-b2df-f480b889d76d-tmp-dir\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j78fw\" (UniqueName: \"kubernetes.io/projected/ecb0f702-58cc-4ffd-b2df-f480b889d76d-kube-api-access-j78fw\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-k8s-cni-cncf-io\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-etc-kubernetes\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-kubernetes\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227675    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-host-run-k8s-cni-cncf-io\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227707    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-kubelet-config\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227730    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-etc-kubernetes\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227793    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-kubelet-config\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227812    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-kubernetes\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.229307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.227837    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9930d01b-c2fb-45eb-950c-c36e2bda9296-konnectivity-ca\") pod \"konnectivity-agent-qvk4s\" (UID: \"9930d01b-c2fb-45eb-950c-c36e2bda9296\") " pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.230064 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.229498    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3fe339e-c4ea-43ec-b999-369382b7f87c-etc-tuned\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.230129 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.230107    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9930d01b-c2fb-45eb-950c-c36e2bda9296-agent-certs\") pod \"konnectivity-agent-qvk4s\" (UID: \"9930d01b-c2fb-45eb-950c-c36e2bda9296\") " pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.230183 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.230148    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3fe339e-c4ea-43ec-b999-369382b7f87c-tmp\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.236036 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.236009    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdrm7\" (UniqueName: \"kubernetes.io/projected/3ac919b2-cc6b-484c-b70e-7e61edf2b63c-kube-api-access-gdrm7\") pod \"multus-fgtff\" (UID: \"3ac919b2-cc6b-484c-b70e-7e61edf2b63c\") " pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.236922 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.236896    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjbs\" (UniqueName: \"kubernetes.io/projected/d3fe339e-c4ea-43ec-b999-369382b7f87c-kube-api-access-xfjbs\") pod \"tuned-gtrlq\" (UID: \"d3fe339e-c4ea-43ec-b999-369382b7f87c\") " pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.238100 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.238065    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h9wc\" (UniqueName: \"kubernetes.io/projected/7b879347-20d5-4742-b1e9-06e4d2ab7870-kube-api-access-2h9wc\") pod \"iptables-alerter-wfhzc\" (UID: \"7b879347-20d5-4742-b1e9-06e4d2ab7870\") " pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.238100 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.238081    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j78fw\" (UniqueName: \"kubernetes.io/projected/ecb0f702-58cc-4ffd-b2df-f480b889d76d-kube-api-access-j78fw\") pod \"node-resolver-b95gn\" (UID: \"ecb0f702-58cc-4ffd-b2df-f480b889d76d\") " pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.319351 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.319303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:10.329632 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.329606    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wsjmn"
Jul 06 23:56:10.340082 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.340055    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs"
Jul 06 23:56:10.346058 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.346032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wdflb"
Jul 06 23:56:10.353294 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.353273    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gtrlq"
Jul 06 23:56:10.359424 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.359399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:10.367447 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.367421    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b95gn"
Jul 06 23:56:10.374427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.374407    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wfhzc"
Jul 06 23:56:10.379442 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.379420    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fgtff"
Jul 06 23:56:10.462610 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.462579    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:10.630684 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.630593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:10.630856 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.630736    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:10.630856 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.630823    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:11.630803808 +0000 UTC m=+4.043817738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:10.731403 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.731358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:10.731610 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:10.731430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:10.731610 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.731596    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:10.731726 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.731654    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:10.731726 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.731680    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:10.731726 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.731699    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:10.731946 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.731684    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:11.731662644 +0000 UTC m=+4.144676547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:10.731946 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:10.731768    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:11.731753331 +0000 UTC m=+4.144767219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:11.087325 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:11.087207    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:10 +0000 UTC" deadline="2028-01-13 21:30:00.537842302 +0000 UTC"
Jul 06 23:56:11.087325 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:11.087274    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13341h33m49.450572337s"
Jul 06 23:56:11.177361 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:11.177216    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac919b2_cc6b_484c_b70e_7e61edf2b63c.slice/crio-d53f6bd44fbf28cd60d7f0b160ddd10deafc26aefb73cb8085169f0a9452cd4a WatchSource:0}: Error finding container d53f6bd44fbf28cd60d7f0b160ddd10deafc26aefb73cb8085169f0a9452cd4a: Status 404 returned error can't find the container with id d53f6bd44fbf28cd60d7f0b160ddd10deafc26aefb73cb8085169f0a9452cd4a
Jul 06 23:56:11.181261 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:11.181234    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef56e8b2_be96_4539_9a30_6bba9c69ddae.slice/crio-c63073d20a0cc73e4dfc253d7877b93d575e412111686c5601d7f3029e82ca4b WatchSource:0}: Error finding container c63073d20a0cc73e4dfc253d7877b93d575e412111686c5601d7f3029e82ca4b: Status 404 returned error can't find the container with id c63073d20a0cc73e4dfc253d7877b93d575e412111686c5601d7f3029e82ca4b
Jul 06 23:56:11.182083 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:11.182058    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9930d01b_c2fb_45eb_950c_c36e2bda9296.slice/crio-71cef01e2fc871cce2e3db4318604ba61034e7b3a7fd8dc143395e0eba028c9d WatchSource:0}: Error finding container 71cef01e2fc871cce2e3db4318604ba61034e7b3a7fd8dc143395e0eba028c9d: Status 404 returned error can't find the container with id 71cef01e2fc871cce2e3db4318604ba61034e7b3a7fd8dc143395e0eba028c9d
Jul 06 23:56:11.189116 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:11.188999    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02710aba_44b6_4d11_8cc3_3e0d2cb50934.slice/crio-1b42abfd364cbdd134d88f728de9faa2207cc09d29689a806c0bde585e0a68df WatchSource:0}: Error finding container 1b42abfd364cbdd134d88f728de9faa2207cc09d29689a806c0bde585e0a68df: Status 404 returned error can't find the container with id 1b42abfd364cbdd134d88f728de9faa2207cc09d29689a806c0bde585e0a68df
Jul 06 23:56:11.191160 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:11.191137    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae9b7ed_cc44_4ca0_9ab2_9850af4e6073.slice/crio-b71bf814d68b7af58cbf5efedc0c1327714b2b08e15349dfd0bba24481495632 WatchSource:0}: Error finding container b71bf814d68b7af58cbf5efedc0c1327714b2b08e15349dfd0bba24481495632: Status 404 returned error can't find the container with id b71bf814d68b7af58cbf5efedc0c1327714b2b08e15349dfd0bba24481495632
Jul 06 23:56:11.639991 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:11.639698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:11.640200 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.640180    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:11.640274 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.640261    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:13.640239618 +0000 UTC m=+6.053253508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:11.741120 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:11.741027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:11.741120 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:11.741097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:11.741359 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.741221    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:11.741359 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.741227    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:11.741359 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.741258    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:11.741359 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.741274    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:11.741359 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.741295    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:13.741276231 +0000 UTC m=+6.154290118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:11.741359 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:11.741333    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:13.741314308 +0000 UTC m=+6.154328213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:12.087834 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.087739    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:10 +0000 UTC" deadline="2028-02-15 23:49:14.265769556 +0000 UTC"
Jul 06 23:56:12.087834 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.087783    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14135h53m2.177990898s"
Jul 06 23:56:12.114784 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.113833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:12.114784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:12.113980    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:12.114784 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.114071    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:12.114784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:12.114156    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:12.114784 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.114611    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:12.114784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:12.114718    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:12.130738 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.130698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerStarted","Data":"b71bf814d68b7af58cbf5efedc0c1327714b2b08e15349dfd0bba24481495632"}
Jul 06 23:56:12.148760 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.148682    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gtrlq" event={"ID":"d3fe339e-c4ea-43ec-b999-369382b7f87c","Type":"ContainerStarted","Data":"8287261aabe2cb96c0e11722b07a56f59524df832aebb65fc9ea639cd54c50e3"}
Jul 06 23:56:12.152446 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.152411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wfhzc" event={"ID":"7b879347-20d5-4742-b1e9-06e4d2ab7870","Type":"ContainerStarted","Data":"d1d4f9aa483aaa095174da9a3b23d9c1f7bc38e99681625b035723ce9668c839"}
Jul 06 23:56:12.158571 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.157782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b95gn" event={"ID":"ecb0f702-58cc-4ffd-b2df-f480b889d76d","Type":"ContainerStarted","Data":"c6f7d2ad242901fb59d33e54ddad3c4ff736b0633e0e70ca94ed681a72ceb1fa"}
Jul 06 23:56:12.171918 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.171872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"3d57778f770b10a6e395304e353fe44b7213accba497642c68cc75adf124cd3c"}
Jul 06 23:56:12.187149 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.187112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvk4s" event={"ID":"9930d01b-c2fb-45eb-950c-c36e2bda9296","Type":"ContainerStarted","Data":"71cef01e2fc871cce2e3db4318604ba61034e7b3a7fd8dc143395e0eba028c9d"}
Jul 06 23:56:12.197987 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.197951    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs" event={"ID":"ef56e8b2-be96-4539-9a30-6bba9c69ddae","Type":"ContainerStarted","Data":"c63073d20a0cc73e4dfc253d7877b93d575e412111686c5601d7f3029e82ca4b"}
Jul 06 23:56:12.210470 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.210154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal" event={"ID":"bc8b346ff26bd2da637db45021bfde3d","Type":"ContainerStarted","Data":"b5640b1f2cc9f43984c814dd2e0c4554f81358d157ead848cb07219bd2871e87"}
Jul 06 23:56:12.216064 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.216025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wsjmn" event={"ID":"02710aba-44b6-4d11-8cc3-3e0d2cb50934","Type":"ContainerStarted","Data":"1b42abfd364cbdd134d88f728de9faa2207cc09d29689a806c0bde585e0a68df"}
Jul 06 23:56:12.222629 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.222556    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fgtff" event={"ID":"3ac919b2-cc6b-484c-b70e-7e61edf2b63c","Type":"ContainerStarted","Data":"d53f6bd44fbf28cd60d7f0b160ddd10deafc26aefb73cb8085169f0a9452cd4a"}
Jul 06 23:56:12.233056 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.233025    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7c519ecbb7e86b3494895d9cd797fde" containerID="67890c4e036b3659a227a41b9dc0f781baa129306429e853ca900cd98f7178aa" exitCode=0
Jul 06 23:56:12.233192 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.233069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal" event={"ID":"b7c519ecbb7e86b3494895d9cd797fde","Type":"ContainerDied","Data":"67890c4e036b3659a227a41b9dc0f781baa129306429e853ca900cd98f7178aa"}
Jul 06 23:56:12.256661 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:12.256178    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-123.ec2.internal" podStartSLOduration=2.25606123 podStartE2EDuration="2.25606123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:12.232164534 +0000 UTC m=+4.645178447" watchObservedRunningTime="2026-07-06 23:56:12.25606123 +0000 UTC m=+4.669075142"
Jul 06 23:56:13.260962 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:13.260923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal" event={"ID":"b7c519ecbb7e86b3494895d9cd797fde","Type":"ContainerStarted","Data":"77f6276eefa8a0f92cf25e9038f41822a9c7f7e922ff7d3fdc11caf86f6b009b"}
Jul 06 23:56:13.662656 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:13.662564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:13.662824 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.662786    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:13.662909 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.662859    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:17.662835356 +0000 UTC m=+10.075849268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:13.763941 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:13.763899    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:13.764124 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:13.763997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:13.764185 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.764152    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:13.764185 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.764173    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:13.764185 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.764187    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:13.764373 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.764249    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:17.764229054 +0000 UTC m=+10.177242944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:13.764373 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.764327    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:13.764373 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:13.764360    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:17.764348943 +0000 UTC m=+10.177362835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:14.108652 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:14.108614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:14.108892 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:14.108810    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:14.109257 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:14.109234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:14.109370 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:14.109348    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:14.109429 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:14.109416    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:14.109513 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:14.109493    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:16.108794 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:16.108754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:16.109296 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:16.108915    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:16.109296 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:16.108944    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:16.109296 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:16.109030    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:16.109296 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:16.109082    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:16.109296 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:16.109145    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:17.698848 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:17.698766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:17.699346 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.698958    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:17.699346 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.699032    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:25.699012039 +0000 UTC m=+18.112025933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:17.799887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:17.799945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.800044    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.800062    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.800082    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.800094    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.800107    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:25.800089615 +0000 UTC m=+18.213103506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:17.800432 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:17.800162    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:25.800149142 +0000 UTC m=+18.213163045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:18.110141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:18.109863    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:18.110141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:18.109974    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:18.110141 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:18.110017    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:18.110141 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:18.110105    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:18.110539 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:18.110351    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:18.110539 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:18.110407    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:20.109417 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:20.109272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:20.109417 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:20.109291    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:20.109417 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:20.109395    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:20.109417 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:20.109417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:20.110041 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:20.109548    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:20.110041 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:20.109635    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:22.108732 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:22.108670    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:22.108732 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:22.108704    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:22.109290 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:22.108854    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:22.109290 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:22.108905    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:22.109290 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:22.109021    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:22.109290 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:22.109113    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:24.109080 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:24.109049    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:24.109558 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:24.109049    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:24.109558 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:24.109204    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:24.109558 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:24.109057    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:24.109558 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:24.109276    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:24.109558 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:24.109316    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:25.757353 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:25.757308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:25.757803 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.757476    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:25.757803 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.757554    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:41.757537004 +0000 UTC m=+34.170550915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:25.858508 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:25.858465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:25.858699 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:25.858580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:25.858699 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.858661    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:25.858813 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.858745    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:41.858724649 +0000 UTC m=+34.271738542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:25.858813 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.858750    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:25.858813 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.858786    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:25.858813 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.858799    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:25.859015 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:25.858846    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:41.858834307 +0000 UTC m=+34.271848193 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:26.108434 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:26.108406    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:26.108434 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:26.108444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:26.108678 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:26.108409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:26.108678 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:26.108516    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:26.108678 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:26.108620    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:26.108825 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:26.108698    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:28.109855 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:28.109821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:28.110294 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:28.109924    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:28.110294 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:28.109942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:28.110294 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:28.110068    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:28.110294 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:28.110119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:28.110294 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:28.110204    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:30.108522 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:30.108486    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:30.109069 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:30.108531    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:30.109069 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:30.108531    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:30.109069 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:30.108619    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:30.109069 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:30.108681    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:30.109069 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:30.108737    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:31.301413 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.300971    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073" containerID="340cf9f7a1dfc53b15cd29db3af17bafc8768025f16cdbdf8efcc0b777b86757" exitCode=0
Jul 06 23:56:31.301413 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.301058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerDied","Data":"340cf9f7a1dfc53b15cd29db3af17bafc8768025f16cdbdf8efcc0b777b86757"}
Jul 06 23:56:31.303282 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.303247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gtrlq" event={"ID":"d3fe339e-c4ea-43ec-b999-369382b7f87c","Type":"ContainerStarted","Data":"a20def23b659a306d2eb1c6248d6cdf337aa79529020338d6ce43e5e5d6d4e1b"}
Jul 06 23:56:31.304788 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.304764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b95gn" event={"ID":"ecb0f702-58cc-4ffd-b2df-f480b889d76d","Type":"ContainerStarted","Data":"1f1b6dd12dafa65e1c9f76737d94a06e6f8cd72841d682cabcda8b3298751656"}
Jul 06 23:56:31.308070 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.308047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"394a02e3ba62f58bdb7fe4544684ede10465a7c4cecca3fca2aeb3e68862b9e0"}
Jul 06 23:56:31.308166 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.308110    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"1014ea7aecfbfa10cd8961e92d86dc1266d53042f2c8e3c682621a43d8819804"}
Jul 06 23:56:31.308166 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.308142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"dfdde72d16f42c91412864624a75c04010147b574bbe61481070ae40f5115ff1"}
Jul 06 23:56:31.308279 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.308188    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"1ce2f93b4f050522be95dcf19ab9b8c7eaa9365e7999162539653e53dd2ac277"}
Jul 06 23:56:31.308279 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.308230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"1c105f62fe1ebf8e145896ffa6df1e13da30531efffd0f99bb2a731ff2c623bd"}
Jul 06 23:56:31.308279 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.308274    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"f3883ec142f788f362f1de09dc2b2b5535582ecd3f804e9f58a2dcba6d563e8b"}
Jul 06 23:56:31.309585 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.309562    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvk4s" event={"ID":"9930d01b-c2fb-45eb-950c-c36e2bda9296","Type":"ContainerStarted","Data":"1a76cd1532e61275cda6931effd57a251af932c9884a13ff2eba778d73d48dc3"}
Jul 06 23:56:31.311184 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.311161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs" event={"ID":"ef56e8b2-be96-4539-9a30-6bba9c69ddae","Type":"ContainerStarted","Data":"191d2f47ea317e5e634803f1b6fde39252b0aef2157113358dc60071eaea8d21"}
Jul 06 23:56:31.312645 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.312623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wsjmn" event={"ID":"02710aba-44b6-4d11-8cc3-3e0d2cb50934","Type":"ContainerStarted","Data":"78b33eebb89fc417fb968dbd0c036706c4719425cba941941f553e114dc57a54"}
Jul 06 23:56:31.315755 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.315726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fgtff" event={"ID":"3ac919b2-cc6b-484c-b70e-7e61edf2b63c","Type":"ContainerStarted","Data":"e51e2e83efbfb3490745d39c667b8daf243f83a88a7cdcda684834919ee7b49e"}
Jul 06 23:56:31.326435 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.326394    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-123.ec2.internal" podStartSLOduration=21.326383589 podStartE2EDuration="21.326383589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:13.280401953 +0000 UTC m=+5.693415867" watchObservedRunningTime="2026-07-06 23:56:31.326383589 +0000 UTC m=+23.739397497"
Jul 06 23:56:31.363908 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.363860    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wsjmn" podStartSLOduration=3.158948237 podStartE2EDuration="22.363845674s" totalImagesPullingTime="19.204897437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:31.34405265 +0000 UTC m=+23.757066559" watchObservedRunningTime="2026-07-06 23:56:31.363845674 +0000 UTC m=+23.776859582"
Jul 06 23:56:31.364096 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.364075    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gtrlq" podStartSLOduration=3.146135333 podStartE2EDuration="22.364070245s" totalImagesPullingTime="19.217934912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:31.363782991 +0000 UTC m=+23.776796913" watchObservedRunningTime="2026-07-06 23:56:31.364070245 +0000 UTC m=+23.777084153"
Jul 06 23:56:31.382144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.382082    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qvk4s" podStartSLOduration=7.519837575 podStartE2EDuration="22.382065765s" totalImagesPullingTime="14.86222819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:31.381907841 +0000 UTC m=+23.794921751" watchObservedRunningTime="2026-07-06 23:56:31.382065765 +0000 UTC m=+23.795079675"
Jul 06 23:56:31.406926 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.406885    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b95gn" podStartSLOduration=3.199184167 podStartE2EDuration="22.40687191s" totalImagesPullingTime="19.207687743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:31.405604487 +0000 UTC m=+23.818618386" watchObservedRunningTime="2026-07-06 23:56:31.40687191 +0000 UTC m=+23.819885796"
Jul 06 23:56:31.431633 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.431598    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fgtff" podStartSLOduration=3.163771291 podStartE2EDuration="22.431586536s" totalImagesPullingTime="19.267815245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:31.431305431 +0000 UTC m=+23.844319339" watchObservedRunningTime="2026-07-06 23:56:31.431586536 +0000 UTC m=+23.844600444"
Jul 06 23:56:31.680162 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:31.680137    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 23:56:32.082472 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.082353    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:56:31.680158793Z","UUID":"0b5a6513-20dd-4441-bb48-1f38f67a9828","Handler":null,"Name":"","Endpoint":""}
Jul 06 23:56:32.084127 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.084100    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 23:56:32.084255 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.084140    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 23:56:32.108413 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.108388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:32.108572 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.108421    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:32.108572 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:32.108524    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:32.108682 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:32.108583    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:32.108682 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.108636    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:32.108761 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:32.108746    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:32.319601 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.319562    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs" event={"ID":"ef56e8b2-be96-4539-9a30-6bba9c69ddae","Type":"ContainerStarted","Data":"386faef1c638423f4038fd4cbc0a32c3007a767d4d29ae854aeb4329e50ca4f9"}
Jul 06 23:56:32.321216 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.321142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wfhzc" event={"ID":"7b879347-20d5-4742-b1e9-06e4d2ab7870","Type":"ContainerStarted","Data":"a3780c2ce4f92bf4b5e85766764e5688f627b44f3efcaba88059fd325681c2c3"}
Jul 06 23:56:32.341687 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:32.341642    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wfhzc" podStartSLOduration=4.128613646 podStartE2EDuration="23.341624414s" totalImagesPullingTime="19.213010768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:32.34162249 +0000 UTC m=+24.754636400" watchObservedRunningTime="2026-07-06 23:56:32.341624414 +0000 UTC m=+24.754638321"
Jul 06 23:56:33.325614 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:33.325368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs" event={"ID":"ef56e8b2-be96-4539-9a30-6bba9c69ddae","Type":"ContainerStarted","Data":"ce9984f434b3e24311c397c6824f402ff95458e92ef65468053cd621b36b88a7"}
Jul 06 23:56:34.108833 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:34.108800    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:34.109048 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:34.108805    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:34.109048 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:34.108946    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:34.109048 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:34.108995    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:34.109211 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:34.109090    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:34.109211 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:34.109170    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:34.330968 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:34.330937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"0a6e9ca1992063026de3e643598c7cc8a9574d0b888a5f3b44b444361814084a"}
Jul 06 23:56:35.992682 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:35.992480    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:35.993106 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:35.993040    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:36.009398 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.007249    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxfxs" podStartSLOduration=5.663910895 podStartE2EDuration="27.007228188s" totalImagesPullingTime="21.343317293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:33.346619224 +0000 UTC m=+25.759633133" watchObservedRunningTime="2026-07-06 23:56:36.007228188 +0000 UTC m=+28.420242097"
Jul 06 23:56:36.109102 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.109071    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:36.109102 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.109088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:36.109281 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:36.109214    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:36.109281 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:36.109258    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:36.109387 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.109317    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:36.109438 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:36.109386    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:36.340046 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.340013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" event={"ID":"38b5994e-7085-4771-b116-8828bdf914ef","Type":"ContainerStarted","Data":"b9a124014af2cf5943092ded775576de38207425b18caceba37016feecfbd4d3"}
Jul 06 23:56:36.340379 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.340359    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:36.341700 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.341680    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073" containerID="2df16a44c39caf0a07c9e0301e76bec1f55a508d9274ede3618f172f42b8f86e" exitCode=0
Jul 06 23:56:36.341818 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.341756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerDied","Data":"2df16a44c39caf0a07c9e0301e76bec1f55a508d9274ede3618f172f42b8f86e"}
Jul 06 23:56:36.341922 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.341905    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:36.342405 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.342306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qvk4s"
Jul 06 23:56:36.355382 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.355364    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:36.368256 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:36.368214    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" podStartSLOduration=8.077982809 podStartE2EDuration="27.368202188s" totalImagesPullingTime="19.290219379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:36.366651734 +0000 UTC m=+28.779665661" watchObservedRunningTime="2026-07-06 23:56:36.368202188 +0000 UTC m=+28.781216096"
Jul 06 23:56:37.344352 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.344328    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 23:56:37.344804 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.344738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:37.361106 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.361081    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wt5k2"]
Jul 06 23:56:37.361254 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.361197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:37.361340 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:37.361315    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:37.361876 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.361859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:37.364427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.364061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ccgjm"]
Jul 06 23:56:37.364427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.364130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:37.364427 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:37.364200    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:37.373641 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.373615    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5rgcf"]
Jul 06 23:56:37.373758 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:37.373743    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:37.373858 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:37.373836    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:38.348710 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:38.348677    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073" containerID="399e396a96a3bdb6985e5c411fdf312261ced54a99e7a964ae41bdffc6e6176e" exitCode=0
Jul 06 23:56:38.349111 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:38.348760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerDied","Data":"399e396a96a3bdb6985e5c411fdf312261ced54a99e7a964ae41bdffc6e6176e"}
Jul 06 23:56:38.349111 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:38.348842    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 23:56:38.453932 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:38.453904    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:56:39.108647 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:39.108472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:39.108797 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:39.108521    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:39.108797 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:39.108710    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:39.108797 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:39.108537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:39.108899 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:39.108794    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:39.108899 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:39.108886    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:40.355132 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:40.355104    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073" containerID="8f97c9ccd9ebee5b5a481db8c3830ebfd375cb1fee8bdd63056ef686a4df2bc3" exitCode=0
Jul 06 23:56:40.355826 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:40.355191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerDied","Data":"8f97c9ccd9ebee5b5a481db8c3830ebfd375cb1fee8bdd63056ef686a4df2bc3"}
Jul 06 23:56:40.376029 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:40.375997    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m" podUID="38b5994e-7085-4771-b116-8828bdf914ef" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 06 23:56:41.108844 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:41.108803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:41.109028 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:41.108912    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:41.109028 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.108925    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:41.109028 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:41.108941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:41.109158 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.109025    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:41.109158 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.109106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:41.782705 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:41.782623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:41.783251 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.782739    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:41.783251 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.782792    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs podName:1df4ec18-0a1d-4039-a90d-e7afc5fca917 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:13.782776748 +0000 UTC m=+66.195790635 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs") pod "network-metrics-daemon-wt5k2" (UID: "1df4ec18-0a1d-4039-a90d-e7afc5fca917") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:41.883424 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:41.883370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:41.883585 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:41.883494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:41.883659 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.883635    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:41.883659 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.883654    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:41.883756 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.883667    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4t4wt for pod openshift-network-diagnostics/network-check-target-ccgjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:41.883756 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.883730    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt podName:4965cddf-5ea8-4899-9bb6-dc5032852c22 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:13.883711864 +0000 UTC m=+66.296725766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4t4wt" (UniqueName: "kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt") pod "network-check-target-ccgjm" (UID: "4965cddf-5ea8-4899-9bb6-dc5032852c22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:41.883914 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.883752    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:41.883914 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:41.883888    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret podName:e044b7a4-d4c4-444c-a225-ffc0f2b5eb15 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:13.883847071 +0000 UTC m=+66.296860973 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret") pod "global-pull-secret-syncer-5rgcf" (UID: "e044b7a4-d4c4-444c-a225-ffc0f2b5eb15") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:43.108394 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.108361    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:43.108394 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.108379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:43.109011 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.108487    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ccgjm" podUID="4965cddf-5ea8-4899-9bb6-dc5032852c22"
Jul 06 23:56:43.109011 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.108507    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:43.109011 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.108580    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wt5k2" podUID="1df4ec18-0a1d-4039-a90d-e7afc5fca917"
Jul 06 23:56:43.109011 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.108665    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5rgcf" podUID="e044b7a4-d4c4-444c-a225-ffc0f2b5eb15"
Jul 06 23:56:43.358334 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.358307    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-123.ec2.internal" event="NodeReady"
Jul 06 23:56:43.358545 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.358492    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 23:56:43.394933 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.394883    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b8fdf9db-9cqph"]
Jul 06 23:56:43.400179 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.400161    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.402991 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.402878    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 23:56:43.402991 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.402908    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 23:56:43.402991 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.402950    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hbkmd\""
Jul 06 23:56:43.403270 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.403058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 23:56:43.408681 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.408661    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 23:56:43.410669 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.410649    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b8fdf9db-9cqph"]
Jul 06 23:56:43.413246 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.413224    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t9ptt"]
Jul 06 23:56:43.416963 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.416948    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m2sjv"]
Jul 06 23:56:43.417091 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.417071    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.419808 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.419792    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 23:56:43.420183 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.420167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf85g\""
Jul 06 23:56:43.420266 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.420228    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.420670 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.420654    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 23:56:43.423533 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.423512    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 23:56:43.423627 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.423533    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 23:56:43.423627 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.423514    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 23:56:43.423742 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.423511    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xpwtc\""
Jul 06 23:56:43.428006 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.427989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m2sjv"]
Jul 06 23:56:43.428788 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.428768    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t9ptt"]
Jul 06 23:56:43.582359 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.582321    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:43.596155 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef761fc-cc8b-4706-a815-25372d7f87e3-tmp-dir\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.596291 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596177    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-registry-certificates\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596291 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f57ae867-009e-4fd5-a888-c66d631c614b-cert\") pod \"ingress-canary-m2sjv\" (UID: \"f57ae867-009e-4fd5-a888-c66d631c614b\") " pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.596399 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-image-registry-private-configuration\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596399 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-trusted-ca\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596399 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-installation-pull-secrets\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596399 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-bound-sa-token\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596574 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jghjh\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-kube-api-access-jghjh\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596574 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.596574 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596499    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef761fc-cc8b-4706-a815-25372d7f87e3-config-volume\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.596574 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bjc8\" (UniqueName: \"kubernetes.io/projected/f57ae867-009e-4fd5-a888-c66d631c614b-kube-api-access-9bjc8\") pod \"ingress-canary-m2sjv\" (UID: \"f57ae867-009e-4fd5-a888-c66d631c614b\") " pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.596574 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596766 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c28759ab-2f45-43d5-a277-1e74916bd272-ca-trust-extracted\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.596766 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.596603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsfn\" (UniqueName: \"kubernetes.io/projected/5ef761fc-cc8b-4706-a815-25372d7f87e3-kube-api-access-zlsfn\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.695713 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.695624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:43.697621 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef761fc-cc8b-4706-a815-25372d7f87e3-config-volume\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.697742 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697642    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bjc8\" (UniqueName: \"kubernetes.io/projected/f57ae867-009e-4fd5-a888-c66d631c614b-kube-api-access-9bjc8\") pod \"ingress-canary-m2sjv\" (UID: \"f57ae867-009e-4fd5-a888-c66d631c614b\") " pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.697849 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.697909 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c28759ab-2f45-43d5-a277-1e74916bd272-ca-trust-extracted\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.697909 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsfn\" (UniqueName: \"kubernetes.io/projected/5ef761fc-cc8b-4706-a815-25372d7f87e3-kube-api-access-zlsfn\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.698007 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef761fc-cc8b-4706-a815-25372d7f87e3-tmp-dir\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.698007 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.697971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-registry-certificates\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698007 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.697992    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:56:43.698144 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.698010    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b8fdf9db-9cqph: secret "image-registry-tls" not found
Jul 06 23:56:43.698144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698009    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f57ae867-009e-4fd5-a888-c66d631c614b-cert\") pod \"ingress-canary-m2sjv\" (UID: \"f57ae867-009e-4fd5-a888-c66d631c614b\") " pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.698144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-image-registry-private-configuration\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698144 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.698070    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls podName:c28759ab-2f45-43d5-a277-1e74916bd272 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:44.198046913 +0000 UTC m=+36.611060817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls") pod "image-registry-7b8fdf9db-9cqph" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272") : secret "image-registry-tls" not found
Jul 06 23:56:43.698144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-trusted-ca\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698144 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-installation-pull-secrets\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-bound-sa-token\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jghjh\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-kube-api-access-jghjh\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.698427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698276    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef761fc-cc8b-4706-a815-25372d7f87e3-config-volume\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.698427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698319    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef761fc-cc8b-4706-a815-25372d7f87e3-tmp-dir\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.698427 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698379    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c28759ab-2f45-43d5-a277-1e74916bd272-ca-trust-extracted\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.698755 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.698567    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:43.698755 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:43.698618    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls podName:5ef761fc-cc8b-4706-a815-25372d7f87e3 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:44.198601333 +0000 UTC m=+36.611615219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls") pod "dns-default-t9ptt" (UID: "5ef761fc-cc8b-4706-a815-25372d7f87e3") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:43.698885 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.698865    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-registry-certificates\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.699418 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.699300    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-trusted-ca\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.702559 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.702400    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-installation-pull-secrets\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.702642 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.702593    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f57ae867-009e-4fd5-a888-c66d631c614b-cert\") pod \"ingress-canary-m2sjv\" (UID: \"f57ae867-009e-4fd5-a888-c66d631c614b\") " pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.703002 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.702980    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-image-registry-private-configuration\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.708105 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.708076    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bjc8\" (UniqueName: \"kubernetes.io/projected/f57ae867-009e-4fd5-a888-c66d631c614b-kube-api-access-9bjc8\") pod \"ingress-canary-m2sjv\" (UID: \"f57ae867-009e-4fd5-a888-c66d631c614b\") " pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.710816 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.710775    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-bound-sa-token\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.711831 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.711814    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsfn\" (UniqueName: \"kubernetes.io/projected/5ef761fc-cc8b-4706-a815-25372d7f87e3-kube-api-access-zlsfn\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:43.712275 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.712250    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jghjh\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-kube-api-access-jghjh\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:43.733756 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.733733    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m2sjv"
Jul 06 23:56:43.866049 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:43.866006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:44.202012 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:44.201981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:44.202469 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:44.202048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:44.202469 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:44.202151    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:44.202469 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:44.202223    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls podName:5ef761fc-cc8b-4706-a815-25372d7f87e3 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:45.202200965 +0000 UTC m=+37.615214874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls") pod "dns-default-t9ptt" (UID: "5ef761fc-cc8b-4706-a815-25372d7f87e3") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:44.202469 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:44.202155    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:56:44.202469 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:44.202252    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b8fdf9db-9cqph: secret "image-registry-tls" not found
Jul 06 23:56:44.202469 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:44.202305    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls podName:c28759ab-2f45-43d5-a277-1e74916bd272 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:45.202290603 +0000 UTC m=+37.615304510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls") pod "image-registry-7b8fdf9db-9cqph" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272") : secret "image-registry-tls" not found
Jul 06 23:56:44.901289 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:44.901264    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m2sjv"]
Jul 06 23:56:44.908606 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:44.908578    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57ae867_009e_4fd5_a888_c66d631c614b.slice/crio-8f1be2fce03f11d5751830d28ea2cce063ef2c47a8699bd1386a22a135e4d9f7 WatchSource:0}: Error finding container 8f1be2fce03f11d5751830d28ea2cce063ef2c47a8699bd1386a22a135e4d9f7: Status 404 returned error can't find the container with id 8f1be2fce03f11d5751830d28ea2cce063ef2c47a8699bd1386a22a135e4d9f7
Jul 06 23:56:45.108866 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.108833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:56:45.108958 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.108833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:56:45.108958 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.108945    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:56:45.111953 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.111937    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:56:45.113088 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.113065    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:56:45.113088 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.113081    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-knf8z\""
Jul 06 23:56:45.113226 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.113087    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lswch\""
Jul 06 23:56:45.113226 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.113107    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:56:45.113226 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.113082    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:56:45.209610 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.209575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:45.209938 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.209654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:45.209938 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:45.209718    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:45.209938 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:45.209772    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls podName:5ef761fc-cc8b-4706-a815-25372d7f87e3 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:47.209753959 +0000 UTC m=+39.622767861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls") pod "dns-default-t9ptt" (UID: "5ef761fc-cc8b-4706-a815-25372d7f87e3") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:45.209938 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:45.209788    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:56:45.209938 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:45.209805    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b8fdf9db-9cqph: secret "image-registry-tls" not found
Jul 06 23:56:45.209938 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:45.209851    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls podName:c28759ab-2f45-43d5-a277-1e74916bd272 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:47.209837086 +0000 UTC m=+39.622850972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls") pod "image-registry-7b8fdf9db-9cqph" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272") : secret "image-registry-tls" not found
Jul 06 23:56:45.367161 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.367131    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m2sjv" event={"ID":"f57ae867-009e-4fd5-a888-c66d631c614b","Type":"ContainerStarted","Data":"8f1be2fce03f11d5751830d28ea2cce063ef2c47a8699bd1386a22a135e4d9f7"}
Jul 06 23:56:45.369838 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.369810    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073" containerID="0fdeaa02402bb71ef6527a55fe3eabdd06fddefb8594d62bae5569b1beb90aa1" exitCode=0
Jul 06 23:56:45.369960 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:45.369875    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerDied","Data":"0fdeaa02402bb71ef6527a55fe3eabdd06fddefb8594d62bae5569b1beb90aa1"}
Jul 06 23:56:46.375747 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:46.375516    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073" containerID="3c46a45b55a42d5d4c7401681e262dadb1c5125480924c961e10f8c2b7dd2155" exitCode=0
Jul 06 23:56:46.376121 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:46.375600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerDied","Data":"3c46a45b55a42d5d4c7401681e262dadb1c5125480924c961e10f8c2b7dd2155"}
Jul 06 23:56:47.221921 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.221852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:47.221921 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.221915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:47.222085 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.222006    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:47.222085 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.222035    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:56:47.222085 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.222049    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b8fdf9db-9cqph: secret "image-registry-tls" not found
Jul 06 23:56:47.222085 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.222079    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls podName:5ef761fc-cc8b-4706-a815-25372d7f87e3 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.222057175 +0000 UTC m=+43.635071082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls") pod "dns-default-t9ptt" (UID: "5ef761fc-cc8b-4706-a815-25372d7f87e3") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:47.222203 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.222102    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls podName:c28759ab-2f45-43d5-a277-1e74916bd272 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.222085686 +0000 UTC m=+43.635099572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls") pod "image-registry-7b8fdf9db-9cqph" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272") : secret "image-registry-tls" not found
Jul 06 23:56:47.383868 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.383838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wdflb" event={"ID":"0ae9b7ed-cc44-4ca0-9ab2-9850af4e6073","Type":"ContainerStarted","Data":"c3e4a09b81d33e0869dfbc72e79faf3f6106090e0c549691284d5f33ec7bf164"}
Jul 06 23:56:47.407894 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.407773    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wdflb" podStartSLOduration=8.463520504 podStartE2EDuration="38.407753615s" totalImagesPullingTime="29.944233111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:47.40765475 +0000 UTC m=+39.820668681" watchObservedRunningTime="2026-07-06 23:56:47.407753615 +0000 UTC m=+39.820767524"
Jul 06 23:56:47.462695 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.462662    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:47.530804 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.530719    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:47.589814 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.589781    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-hkx2q"]
Jul 06 23:56:47.592909 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.592882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:47.612208 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.612180    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-hkx2q"]
Jul 06 23:56:47.612310 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.612229    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.615442 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.615420    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 06 23:56:47.615442 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.615438    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 06 23:56:47.615605 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.615423    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.615605 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.615502    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tb7cc\""
Jul 06 23:56:47.615792 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.615774    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.621077 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.621061    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 06 23:56:47.625410 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.625393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.625492 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.625425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982fh\" (UniqueName: \"kubernetes.io/projected/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-kube-api-access-982fh\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.625492 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.625449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-snapshots\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.625600 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.625547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.625600 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.625581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-tmp\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.625670 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.625606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-serving-cert\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.700122 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.700096    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"]
Jul 06 23:56:47.726317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.726210    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"]
Jul 06 23:56:47.726581 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.726419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.726581 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.726430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.727530 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.727475    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.727891 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.727550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-tmp\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.727891 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.727686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-serving-cert\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.728139 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.728034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.728339 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.728158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982fh\" (UniqueName: \"kubernetes.io/projected/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-kube-api-access-982fh\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.728879 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.728389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-snapshots\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.731477 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.731442    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 06 23:56:47.731621 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.731593    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.731703 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.731550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 06 23:56:47.731703 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.731672    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.731782 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.731749    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.732401 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.732382    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vl48p\""
Jul 06 23:56:47.744095 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.744076    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-tmp\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.744294 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.744279    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-snapshots\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.747528 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.747510    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-serving-cert\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.748211 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.748194    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"]
Jul 06 23:56:47.748312 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.748298    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.752829 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.752810    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 06 23:56:47.752940 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.752859    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.757113 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.757093    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mmgpt\""
Jul 06 23:56:47.760948 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.760932    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.761552 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.761538    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 06 23:56:47.762426 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.762409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"]
Jul 06 23:56:47.762541 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.762435    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"]
Jul 06 23:56:47.762541 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.762528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"
Jul 06 23:56:47.764964 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.764937    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-982fh\" (UniqueName: \"kubernetes.io/projected/07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc-kube-api-access-982fh\") pod \"insights-operator-5c847dcfc4-hkx2q\" (UID: \"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc\") " pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.765557 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.765544    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h676f\""
Jul 06 23:56:47.776522 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.776506    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68846f8cb5-nx52d"]
Jul 06 23:56:47.776620 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.776607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.780005 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.779990    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 06 23:56:47.780394 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.780381    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.780735 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.780671    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zckl5\""
Jul 06 23:56:47.780830 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.780676    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.782764 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.782744    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 06 23:56:47.788719 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.788701    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"]
Jul 06 23:56:47.788810 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.788726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"]
Jul 06 23:56:47.788810 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.788739    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"]
Jul 06 23:56:47.788810 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.788751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68846f8cb5-nx52d"]
Jul 06 23:56:47.788901 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.788812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.793076 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.793058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 06 23:56:47.793171 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.793132    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 06 23:56:47.793373 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.793355    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-njrfs\""
Jul 06 23:56:47.793635 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.793615    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.793783 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.793763    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 06 23:56:47.793900 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.793783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 06 23:56:47.795516 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.795467    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.796241 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.796224    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"]
Jul 06 23:56:47.813137 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.813117    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-cht2h"]
Jul 06 23:56:47.813248 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.813232    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"
Jul 06 23:56:47.816545 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.816525    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.816545 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.816543    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.816804 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.816792    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tnpwl\""
Jul 06 23:56:47.823269 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.823251    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"]
Jul 06 23:56:47.823348 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.823284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-cht2h"]
Jul 06 23:56:47.823348 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.823344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.826298 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.826282    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 06 23:56:47.826298 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.826291    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.826437 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.826293    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dlt9c\""
Jul 06 23:56:47.826437 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.826326    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.826773 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.826759    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 06 23:56:47.829364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829344    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t55ff\" (UniqueName: \"kubernetes.io/projected/f059a428-7a21-4caf-a2e5-d0f726632bc9-kube-api-access-t55ff\") pod \"volume-data-source-validator-5d456fdbcd-zbc8b\" (UID: \"f059a428-7a21-4caf-a2e5-d0f726632bc9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"
Jul 06 23:56:47.829497 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829481    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.829560 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/541e45ab-dde2-48dc-b018-b187bcf1387a-serving-cert\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.829800 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/541e45ab-dde2-48dc-b018-b187bcf1387a-config\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.829800 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/637b2feb-ed55-4e15-889e-9d8eb9becc51-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.829800 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/061a0d8e-ce66-4d25-b524-839070cbd53c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.829800 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9g9w\" (UniqueName: \"kubernetes.io/projected/7176dbf9-7969-411f-9efb-dec85745eed8-kube-api-access-c9g9w\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.829800 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7176dbf9-7969-411f-9efb-dec85745eed8-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829834    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/637b2feb-ed55-4e15-889e-9d8eb9becc51-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829866    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxtv\" (UniqueName: \"kubernetes.io/projected/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-kube-api-access-jgxtv\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/541e45ab-dde2-48dc-b018-b187bcf1387a-trusted-ca\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829919    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58t5h\" (UniqueName: \"kubernetes.io/projected/c9f0f22d-8168-48d7-b56d-ba6098d5cde5-kube-api-access-58t5h\") pod \"network-check-source-989f8b8dc-jlv7l\" (UID: \"c9f0f22d-8168-48d7-b56d-ba6098d5cde5\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.829974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdl4d\" (UniqueName: \"kubernetes.io/projected/637b2feb-ed55-4e15-889e-9d8eb9becc51-kube-api-access-zdl4d\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.830001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-default-certificate\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.830027    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-stats-auth\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.830054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2mx7\" (UniqueName: \"kubernetes.io/projected/541e45ab-dde2-48dc-b018-b187bcf1387a-kube-api-access-x2mx7\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.830094    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhz7n\" (UniqueName: \"kubernetes.io/projected/061a0d8e-ce66-4d25-b524-839070cbd53c-kube-api-access-xhz7n\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.830139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7176dbf9-7969-411f-9efb-dec85745eed8-config\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.830187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.830162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.833054 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.833036    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 06 23:56:47.906218 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.906192    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"]
Jul 06 23:56:47.911663 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.911646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:47.914449 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.914389    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 06 23:56:47.914449 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.914438    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:47.914693 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.914597    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:47.914917 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.914902    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wqqfm\""
Jul 06 23:56:47.921322 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.921306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q"
Jul 06 23:56:47.923527 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.923508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"]
Jul 06 23:56:47.931177 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2mx7\" (UniqueName: \"kubernetes.io/projected/541e45ab-dde2-48dc-b018-b187bcf1387a-kube-api-access-x2mx7\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.931252 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhz7n\" (UniqueName: \"kubernetes.io/projected/061a0d8e-ce66-4d25-b524-839070cbd53c-kube-api-access-xhz7n\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.931252 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7176dbf9-7969-411f-9efb-dec85745eed8-config\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.931252 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.931408 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t55ff\" (UniqueName: \"kubernetes.io/projected/f059a428-7a21-4caf-a2e5-d0f726632bc9-kube-api-access-t55ff\") pod \"volume-data-source-validator-5d456fdbcd-zbc8b\" (UID: \"f059a428-7a21-4caf-a2e5-d0f726632bc9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"
Jul 06 23:56:47.931408 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931329    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.931408 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/541e45ab-dde2-48dc-b018-b187bcf1387a-serving-cert\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.931408 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.931387    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.43136468 +0000 UTC m=+40.844378583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : configmap references non-existent config key: service-ca.crt
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/541e45ab-dde2-48dc-b018-b187bcf1387a-config\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.931472    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/637b2feb-ed55-4e15-889e-9d8eb9becc51-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/061a0d8e-ce66-4d25-b524-839070cbd53c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.931528    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.431513671 +0000 UTC m=+40.844527579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : secret "router-metrics-certs-default" not found
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9g9w\" (UniqueName: \"kubernetes.io/projected/7176dbf9-7969-411f-9efb-dec85745eed8-kube-api-access-c9g9w\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7176dbf9-7969-411f-9efb-dec85745eed8-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.931664 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:47.932043 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931795    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7176dbf9-7969-411f-9efb-dec85745eed8-config\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.932043 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.931979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rblx\" (UniqueName: \"kubernetes.io/projected/381a88ba-f473-4669-8f6a-6e67289d9c8c-kube-api-access-7rblx\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:47.932151 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/637b2feb-ed55-4e15-889e-9d8eb9becc51-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.932151 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932067    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/541e45ab-dde2-48dc-b018-b187bcf1387a-config\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.932151 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxtv\" (UniqueName: \"kubernetes.io/projected/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-kube-api-access-jgxtv\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.932151 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/541e45ab-dde2-48dc-b018-b187bcf1387a-trusted-ca\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.932151 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.932381 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58t5h\" (UniqueName: \"kubernetes.io/projected/c9f0f22d-8168-48d7-b56d-ba6098d5cde5-kube-api-access-58t5h\") pod \"network-check-source-989f8b8dc-jlv7l\" (UID: \"c9f0f22d-8168-48d7-b56d-ba6098d5cde5\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"
Jul 06 23:56:47.932381 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdl4d\" (UniqueName: \"kubernetes.io/projected/637b2feb-ed55-4e15-889e-9d8eb9becc51-kube-api-access-zdl4d\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.932381 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-default-certificate\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.932381 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932230    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/061a0d8e-ce66-4d25-b524-839070cbd53c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.932381 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-stats-auth\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.932381 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.932352    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:47.932713 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:47.932397    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls podName:061a0d8e-ce66-4d25-b524-839070cbd53c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.432383352 +0000 UTC m=+40.845397256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-djml9" (UID: "061a0d8e-ce66-4d25-b524-839070cbd53c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:47.933693 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.932803    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/637b2feb-ed55-4e15-889e-9d8eb9becc51-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.933693 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.933357    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/541e45ab-dde2-48dc-b018-b187bcf1387a-trusted-ca\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.934591 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.934407    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/637b2feb-ed55-4e15-889e-9d8eb9becc51-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.934699 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.934621    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/541e45ab-dde2-48dc-b018-b187bcf1387a-serving-cert\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.934699 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.934633    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7176dbf9-7969-411f-9efb-dec85745eed8-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.935091 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.935073    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-default-certificate\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.935470 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.935428    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-stats-auth\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.945972 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.945944    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9g9w\" (UniqueName: \"kubernetes.io/projected/7176dbf9-7969-411f-9efb-dec85745eed8-kube-api-access-c9g9w\") pod \"service-ca-operator-5d46d9ffd4-dmldm\" (UID: \"7176dbf9-7969-411f-9efb-dec85745eed8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:47.946073 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.945953    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2mx7\" (UniqueName: \"kubernetes.io/projected/541e45ab-dde2-48dc-b018-b187bcf1387a-kube-api-access-x2mx7\") pod \"console-operator-68b757fb67-cht2h\" (UID: \"541e45ab-dde2-48dc-b018-b187bcf1387a\") " pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:47.946073 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.946007    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t55ff\" (UniqueName: \"kubernetes.io/projected/f059a428-7a21-4caf-a2e5-d0f726632bc9-kube-api-access-t55ff\") pod \"volume-data-source-validator-5d456fdbcd-zbc8b\" (UID: \"f059a428-7a21-4caf-a2e5-d0f726632bc9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"
Jul 06 23:56:47.946203 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.946183    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxtv\" (UniqueName: \"kubernetes.io/projected/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-kube-api-access-jgxtv\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:47.946498 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.946479    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58t5h\" (UniqueName: \"kubernetes.io/projected/c9f0f22d-8168-48d7-b56d-ba6098d5cde5-kube-api-access-58t5h\") pod \"network-check-source-989f8b8dc-jlv7l\" (UID: \"c9f0f22d-8168-48d7-b56d-ba6098d5cde5\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"
Jul 06 23:56:47.947119 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.947102    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhz7n\" (UniqueName: \"kubernetes.io/projected/061a0d8e-ce66-4d25-b524-839070cbd53c-kube-api-access-xhz7n\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:47.947761 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.947738    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdl4d\" (UniqueName: \"kubernetes.io/projected/637b2feb-ed55-4e15-889e-9d8eb9becc51-kube-api-access-zdl4d\") pod \"kube-storage-version-migrator-operator-846bf668f7-rppns\" (UID: \"637b2feb-ed55-4e15-889e-9d8eb9becc51\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:47.975419 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:47.975324    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:48.033541 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.033470    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:48.033541 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.033519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rblx\" (UniqueName: \"kubernetes.io/projected/381a88ba-f473-4669-8f6a-6e67289d9c8c-kube-api-access-7rblx\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:48.033691 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.033575    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:56:48.033691 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.033637    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls podName:381a88ba-f473-4669-8f6a-6e67289d9c8c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:48.533622619 +0000 UTC m=+40.946636524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qs6g9" (UID: "381a88ba-f473-4669-8f6a-6e67289d9c8c") : secret "samples-operator-tls" not found
Jul 06 23:56:48.038395 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.038335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"
Jul 06 23:56:48.041957 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.041939    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rblx\" (UniqueName: \"kubernetes.io/projected/381a88ba-f473-4669-8f6a-6e67289d9c8c-kube-api-access-7rblx\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:48.047320 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.047247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-hkx2q"]
Jul 06 23:56:48.049598 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:48.049576    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b8dee1_5f43_4ef5_bc2f_d83ccbc725cc.slice/crio-9d06df26d36572a93a70f653a2201434fdbaade9b8181a66b7e84dd4f470dd45 WatchSource:0}: Error finding container 9d06df26d36572a93a70f653a2201434fdbaade9b8181a66b7e84dd4f470dd45: Status 404 returned error can't find the container with id 9d06df26d36572a93a70f653a2201434fdbaade9b8181a66b7e84dd4f470dd45
Jul 06 23:56:48.050537 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.050496    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:48.059630 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.059611    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"
Jul 06 23:56:48.073831 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.073806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"
Jul 06 23:56:48.120805 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.120785    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"
Jul 06 23:56:48.132512 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.132441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:48.232595 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.232524    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm"]
Jul 06 23:56:48.240832 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:48.240602    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7176dbf9_7969_411f_9efb_dec85745eed8.slice/crio-dcdd95e4cadc5cbb2cfd64e935e6479c53f7eb31dfc84e3c36959f09821d00c1 WatchSource:0}: Error finding container dcdd95e4cadc5cbb2cfd64e935e6479c53f7eb31dfc84e3c36959f09821d00c1: Status 404 returned error can't find the container with id dcdd95e4cadc5cbb2cfd64e935e6479c53f7eb31dfc84e3c36959f09821d00c1
Jul 06 23:56:48.250925 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.250856    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns"]
Jul 06 23:56:48.255684 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:48.255643    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637b2feb_ed55_4e15_889e_9d8eb9becc51.slice/crio-db88332a1a93da4e856a4f5d3868c7ee1d2f0b15d230b83e5c0e3ca2caa5ca37 WatchSource:0}: Error finding container db88332a1a93da4e856a4f5d3868c7ee1d2f0b15d230b83e5c0e3ca2caa5ca37: Status 404 returned error can't find the container with id db88332a1a93da4e856a4f5d3868c7ee1d2f0b15d230b83e5c0e3ca2caa5ca37
Jul 06 23:56:48.285109 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.285089    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l"]
Jul 06 23:56:48.287818 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:48.287796    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f0f22d_8168_48d7_b56d_ba6098d5cde5.slice/crio-8ed95c8461ef34d93e3b4e3cf4211b44137e8852a52b9f912cf0bcc2b430ee62 WatchSource:0}: Error finding container 8ed95c8461ef34d93e3b4e3cf4211b44137e8852a52b9f912cf0bcc2b430ee62: Status 404 returned error can't find the container with id 8ed95c8461ef34d93e3b4e3cf4211b44137e8852a52b9f912cf0bcc2b430ee62
Jul 06 23:56:48.307951 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.307925    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b"]
Jul 06 23:56:48.309629 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:56:48.309599    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf059a428_7a21_4caf_a2e5_d0f726632bc9.slice/crio-97a71b9b5c4b34acce056c9e52bdb23b23aae580976dd2f8f0fe6565ad7df96a WatchSource:0}: Error finding container 97a71b9b5c4b34acce056c9e52bdb23b23aae580976dd2f8f0fe6565ad7df96a: Status 404 returned error can't find the container with id 97a71b9b5c4b34acce056c9e52bdb23b23aae580976dd2f8f0fe6565ad7df96a
Jul 06 23:56:48.335314 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.335286    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-cht2h"]
Jul 06 23:56:48.387091 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.387059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b" event={"ID":"f059a428-7a21-4caf-a2e5-d0f726632bc9","Type":"ContainerStarted","Data":"97a71b9b5c4b34acce056c9e52bdb23b23aae580976dd2f8f0fe6565ad7df96a"}
Jul 06 23:56:48.388127 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.388103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" event={"ID":"541e45ab-dde2-48dc-b018-b187bcf1387a","Type":"ContainerStarted","Data":"eddae3ccdb840893b2652979df4e7d67a8998b868bac904eb721cc41777fb0b9"}
Jul 06 23:56:48.389054 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.389035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l" event={"ID":"c9f0f22d-8168-48d7-b56d-ba6098d5cde5","Type":"ContainerStarted","Data":"8ed95c8461ef34d93e3b4e3cf4211b44137e8852a52b9f912cf0bcc2b430ee62"}
Jul 06 23:56:48.390027 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.390005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q" event={"ID":"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc","Type":"ContainerStarted","Data":"9d06df26d36572a93a70f653a2201434fdbaade9b8181a66b7e84dd4f470dd45"}
Jul 06 23:56:48.390998 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.390982    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns" event={"ID":"637b2feb-ed55-4e15-889e-9d8eb9becc51","Type":"ContainerStarted","Data":"db88332a1a93da4e856a4f5d3868c7ee1d2f0b15d230b83e5c0e3ca2caa5ca37"}
Jul 06 23:56:48.391924 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.391903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm" event={"ID":"7176dbf9-7969-411f-9efb-dec85745eed8","Type":"ContainerStarted","Data":"dcdd95e4cadc5cbb2cfd64e935e6479c53f7eb31dfc84e3c36959f09821d00c1"}
Jul 06 23:56:48.393112 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.393092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m2sjv" event={"ID":"f57ae867-009e-4fd5-a888-c66d631c614b","Type":"ContainerStarted","Data":"bf34c100a8c4706071e2497038920c07118f8aeee03afca432c1ec673dc248b9"}
Jul 06 23:56:48.412228 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.412192    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m2sjv" podStartSLOduration=3.042747234 podStartE2EDuration="5.412179556s" totalImagesPullingTime="2.369432322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:48.411509008 +0000 UTC m=+40.824522935" watchObservedRunningTime="2026-07-06 23:56:48.412179556 +0000 UTC m=+40.825193517"
Jul 06 23:56:48.436202 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.436169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:48.436308 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.436236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:48.436768 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.436313    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:48.436768 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.436344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:48.436768 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.436393    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls podName:061a0d8e-ce66-4d25-b524-839070cbd53c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.436372342 +0000 UTC m=+41.849386244 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-djml9" (UID: "061a0d8e-ce66-4d25-b524-839070cbd53c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:48.436768 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.436430    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:56:48.436768 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.436434    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.436412455 +0000 UTC m=+41.849426354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : configmap references non-existent config key: service-ca.crt
Jul 06 23:56:48.436768 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.436502    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.43648939 +0000 UTC m=+41.849503283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : secret "router-metrics-certs-default" not found
Jul 06 23:56:48.537225 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:48.537155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:48.537356 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.537262    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:56:48.537356 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:48.537323    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls podName:381a88ba-f473-4669-8f6a-6e67289d9c8c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:49.537308317 +0000 UTC m=+41.950322204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qs6g9" (UID: "381a88ba-f473-4669-8f6a-6e67289d9c8c") : secret "samples-operator-tls" not found
Jul 06 23:56:49.448803 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:49.448764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:49.449234 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:49.448903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:49.449234 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:49.448973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:49.449234 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.449115    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.449097556 +0000 UTC m=+43.862111458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : configmap references non-existent config key: service-ca.crt
Jul 06 23:56:49.449520 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.449502    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:49.449587 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.449557    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls podName:061a0d8e-ce66-4d25-b524-839070cbd53c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.449542369 +0000 UTC m=+43.862556257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-djml9" (UID: "061a0d8e-ce66-4d25-b524-839070cbd53c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:49.449644 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.449608    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:56:49.449644 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.449639    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.449626372 +0000 UTC m=+43.862640262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : secret "router-metrics-certs-default" not found
Jul 06 23:56:49.550322 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:49.550242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:49.550792 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.550518    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:56:49.550792 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:49.550585    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls podName:381a88ba-f473-4669-8f6a-6e67289d9c8c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:51.550565571 +0000 UTC m=+43.963579483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qs6g9" (UID: "381a88ba-f473-4669-8f6a-6e67289d9c8c") : secret "samples-operator-tls" not found
Jul 06 23:56:51.266701 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:51.266661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:51.267101 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.266804    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:56:51.267101 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.266822    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b8fdf9db-9cqph: secret "image-registry-tls" not found
Jul 06 23:56:51.267101 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:51.266842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:51.267101 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.266874    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls podName:c28759ab-2f45-43d5-a277-1e74916bd272 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:59.266859266 +0000 UTC m=+51.679873152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls") pod "image-registry-7b8fdf9db-9cqph" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272") : secret "image-registry-tls" not found
Jul 06 23:56:51.267101 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.266917    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:51.267101 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.266976    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls podName:5ef761fc-cc8b-4706-a815-25372d7f87e3 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:59.266963553 +0000 UTC m=+51.679977440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls") pod "dns-default-t9ptt" (UID: "5ef761fc-cc8b-4706-a815-25372d7f87e3") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:51.468272 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:51.468241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:51.468417 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:51.468284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:51.468417 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:51.468330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:51.468417 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.468377    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:51.468522 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.468432    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls podName:061a0d8e-ce66-4d25-b524-839070cbd53c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:55.468416876 +0000 UTC m=+47.881430763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-djml9" (UID: "061a0d8e-ce66-4d25-b524-839070cbd53c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:51.468522 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.468465    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:56:51.468522 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.468466    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:55.468437793 +0000 UTC m=+47.881451680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : configmap references non-existent config key: service-ca.crt
Jul 06 23:56:51.468522 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.468511    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:55.46850165 +0000 UTC m=+47.881515536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : secret "router-metrics-certs-default" not found
Jul 06 23:56:51.569547 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:51.569476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:51.569684 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.569646    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:56:51.569737 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:51.569723    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls podName:381a88ba-f473-4669-8f6a-6e67289d9c8c nodeName:}" failed. No retries permitted until 2026-07-06 23:56:55.569704234 +0000 UTC m=+47.982718215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qs6g9" (UID: "381a88ba-f473-4669-8f6a-6e67289d9c8c") : secret "samples-operator-tls" not found
Jul 06 23:56:54.338042 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:54.338016    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:54.411420 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:54.411395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b" event={"ID":"f059a428-7a21-4caf-a2e5-d0f726632bc9","Type":"ContainerStarted","Data":"706d9c366533f1f046aa4058da9ac12c6196ac71ea7264b6a565ef8074a66d34"}
Jul 06 23:56:54.439177 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:54.439136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:55.417312 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.417281    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/0.log"
Jul 06 23:56:55.417791 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.417327    2564 generic.go:356] "Generic (PLEG): container finished" podID="541e45ab-dde2-48dc-b018-b187bcf1387a" containerID="11baf47006cf3ce2b2bcc1462e1de4dc2881f581c567e74e2948e13bee2f80b8" exitCode=255
Jul 06 23:56:55.417791 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.417412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" event={"ID":"541e45ab-dde2-48dc-b018-b187bcf1387a","Type":"ContainerDied","Data":"11baf47006cf3ce2b2bcc1462e1de4dc2881f581c567e74e2948e13bee2f80b8"}
Jul 06 23:56:55.417923 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.417894    2564 scope.go:117] "RemoveContainer" containerID="11baf47006cf3ce2b2bcc1462e1de4dc2881f581c567e74e2948e13bee2f80b8"
Jul 06 23:56:55.419218 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.419185    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l" event={"ID":"c9f0f22d-8168-48d7-b56d-ba6098d5cde5","Type":"ContainerStarted","Data":"4bc8f0cd4bac8ddbfde6ff86e6cdea94f8c20c5a2df3646f2c90b940b4cc78bd"}
Jul 06 23:56:55.421233 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.421206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q" event={"ID":"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc","Type":"ContainerStarted","Data":"a9af6b516cb9ec4fbf15090f79e5b1595108ca2255ab22915730874079f45d1b"}
Jul 06 23:56:55.422926 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.422903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns" event={"ID":"637b2feb-ed55-4e15-889e-9d8eb9becc51","Type":"ContainerStarted","Data":"047b5787f29554a21e002b080a4a9127718fec672f2b641cce0f5d12ea1e1b88"}
Jul 06 23:56:55.424480 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.424424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm" event={"ID":"7176dbf9-7969-411f-9efb-dec85745eed8","Type":"ContainerStarted","Data":"c5d60fb3976735bfa0dec394ae264acf305ba6d1a2f9ef149a53caa2483637bb"}
Jul 06 23:56:55.464495 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.464430    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-zbc8b" podStartSLOduration=2.51576682 podStartE2EDuration="8.464409925s" totalImagesPullingTime="5.948643105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.464035115 +0000 UTC m=+47.877049025" watchObservedRunningTime="2026-07-06 23:56:55.464409925 +0000 UTC m=+47.877423837"
Jul 06 23:56:55.491278 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.491227    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q" podStartSLOduration=2.262697104 podStartE2EDuration="8.491210822s" totalImagesPullingTime="6.228513718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.489904334 +0000 UTC m=+47.902918243" watchObservedRunningTime="2026-07-06 23:56:55.491210822 +0000 UTC m=+47.904224731"
Jul 06 23:56:55.505061 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.505035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:56:55.505187 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.505145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:55.505285 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.505270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:56:55.506259 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.506239    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:03.506218431 +0000 UTC m=+55.919232331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : configmap references non-existent config key: service-ca.crt
Jul 06 23:56:55.507086 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.507069    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:56:55.507086 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.507080    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:55.507247 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.507119    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:03.507102956 +0000 UTC m=+55.920116862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : secret "router-metrics-certs-default" not found
Jul 06 23:56:55.507247 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.507136    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls podName:061a0d8e-ce66-4d25-b524-839070cbd53c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:03.507126967 +0000 UTC m=+55.920140854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-djml9" (UID: "061a0d8e-ce66-4d25-b524-839070cbd53c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:56:55.524486 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.524424    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-jlv7l" podStartSLOduration=2.184344524 podStartE2EDuration="8.52440738s" totalImagesPullingTime="6.340062856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.524155806 +0000 UTC m=+47.937169740" watchObservedRunningTime="2026-07-06 23:56:55.52440738 +0000 UTC m=+47.937421288"
Jul 06 23:56:55.572972 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.572918    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm" podStartSLOduration=2.529125166 podStartE2EDuration="8.572901848s" totalImagesPullingTime="6.043776682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.570735705 +0000 UTC m=+47.983749614" watchObservedRunningTime="2026-07-06 23:56:55.572901848 +0000 UTC m=+47.985915777"
Jul 06 23:56:55.606898 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.606860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:56:55.607055 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.607038    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:56:55.607147 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:55.607112    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls podName:381a88ba-f473-4669-8f6a-6e67289d9c8c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:03.607091112 +0000 UTC m=+56.020104999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qs6g9" (UID: "381a88ba-f473-4669-8f6a-6e67289d9c8c") : secret "samples-operator-tls" not found
Jul 06 23:56:55.615692 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.615649    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns" podStartSLOduration=2.594722453 podStartE2EDuration="8.615634953s" totalImagesPullingTime="6.0209125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:55.613646309 +0000 UTC m=+48.026660460" watchObservedRunningTime="2026-07-06 23:56:55.615634953 +0000 UTC m=+48.028648860"
Jul 06 23:56:55.665888 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.665790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:55.765856 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:55.765822    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:56.001894 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.001858    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r875p"]
Jul 06 23:56:56.019371 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.019335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.024665 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.024642    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 23:56:56.024778 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.024711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q9bzd\""
Jul 06 23:56:56.025543 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.025518    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 23:56:56.027884 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.027867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r875p"]
Jul 06 23:56:56.050656 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.050631    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:56.112130 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.112104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6vjj\" (UniqueName: \"kubernetes.io/projected/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-api-access-b6vjj\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.112249 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.112142    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba836-686a-46b1-94e4-b5a4fab82637-data-volume\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.112249 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.112188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.112352 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.112285    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.112352 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.112308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e1ba836-686a-46b1-94e4-b5a4fab82637-crio-socket\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.212890 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.212855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6vjj\" (UniqueName: \"kubernetes.io/projected/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-api-access-b6vjj\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.213061 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.212963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba836-686a-46b1-94e4-b5a4fab82637-data-volume\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.213061 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.213011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.213177 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.213093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.213177 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.213123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e1ba836-686a-46b1-94e4-b5a4fab82637-crio-socket\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.213271 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.213190    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:56.213310 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.213271    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:56.713250649 +0000 UTC m=+49.126264538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:56.213310 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.213279    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:56.213406 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.213333    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:56.713316369 +0000 UTC m=+49.126330254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:56.213406 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.213395    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e1ba836-686a-46b1-94e4-b5a4fab82637-crio-socket\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.213530 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.213398    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba836-686a-46b1-94e4-b5a4fab82637-data-volume\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.231250 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.231223    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6vjj\" (UniqueName: \"kubernetes.io/projected/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-api-access-b6vjj\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.250618 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.250590    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:56.359372 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.359342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:56.429573 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.429548    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/1.log"
Jul 06 23:56:56.430005 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.429989    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/0.log"
Jul 06 23:56:56.430081 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.430028    2564 generic.go:356] "Generic (PLEG): container finished" podID="541e45ab-dde2-48dc-b018-b187bcf1387a" containerID="d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b" exitCode=255
Jul 06 23:56:56.430166 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.430138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" event={"ID":"541e45ab-dde2-48dc-b018-b187bcf1387a","Type":"ContainerDied","Data":"d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b"}
Jul 06 23:56:56.430221 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.430206    2564 scope.go:117] "RemoveContainer" containerID="11baf47006cf3ce2b2bcc1462e1de4dc2881f581c567e74e2948e13bee2f80b8"
Jul 06 23:56:56.430370 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.430351    2564 scope.go:117] "RemoveContainer" containerID="d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b"
Jul 06 23:56:56.430589 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.430568    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-cht2h_openshift-console-operator(541e45ab-dde2-48dc-b018-b187bcf1387a)\"" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podUID="541e45ab-dde2-48dc-b018-b187bcf1387a"
Jul 06 23:56:56.510637 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.510611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:56.683412 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.683307    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:56.718613 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.718587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.718733 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.718664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:56.718770 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.718739    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:56.718770 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.718747    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:56.718853 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.718807    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:57.718783383 +0000 UTC m=+50.131797273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:56.718853 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:56.718823    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:57.718816343 +0000 UTC m=+50.131830232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:56.876140 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:56.876103    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:57.434640 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:57.434606    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/1.log"
Jul 06 23:56:57.435008 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:57.434926    2564 scope.go:117] "RemoveContainer" containerID="d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b"
Jul 06 23:56:57.435108 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:57.435089    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-cht2h_openshift-console-operator(541e45ab-dde2-48dc-b018-b187bcf1387a)\"" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podUID="541e45ab-dde2-48dc-b018-b187bcf1387a"
Jul 06 23:56:57.456199 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:57.456175    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b95gn_ecb0f702-58cc-4ffd-b2df-f480b889d76d/dns-node-resolver/0.log"
Jul 06 23:56:57.728362 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:57.728281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:57.728521 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:57.728294    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:57.728521 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:57.728417    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:57.728521 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:57.728442    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:59.72842237 +0000 UTC m=+52.141436279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:57.728521 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:57.728375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:57.728521 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:57.728476    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:59.728465223 +0000 UTC m=+52.141479125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:58.000817 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.000740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:58.063700 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.063673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:58.132772 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.132741    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:58.132932 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.132797    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:56:58.219992 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.219956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:58.283105 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.283027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:58.437756 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.437731    2564 scope.go:117] "RemoveContainer" containerID="d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b"
Jul 06 23:56:58.438106 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:58.437875    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-cht2h_openshift-console-operator(541e45ab-dde2-48dc-b018-b187bcf1387a)\"" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podUID="541e45ab-dde2-48dc-b018-b187bcf1387a"
Jul 06 23:56:58.855272 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.855246    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wsjmn_02710aba-44b6-4d11-8cc3-3e0d2cb50934/node-ca/0.log"
Jul 06 23:56:58.870301 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.870271    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:58.968159 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:58.968123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:59.070838 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.070803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:59.342433 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.342408    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:56:59.342591 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.342554    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:56:59.342591 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.342571    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b8fdf9db-9cqph: secret "image-registry-tls" not found
Jul 06 23:56:59.342591 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.342568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:56:59.342702 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.342619    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls podName:c28759ab-2f45-43d5-a277-1e74916bd272 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:15.34260353 +0000 UTC m=+67.755617417 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls") pod "image-registry-7b8fdf9db-9cqph" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272") : secret "image-registry-tls" not found
Jul 06 23:56:59.342702 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.342673    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:56:59.342769 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.342724    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls podName:5ef761fc-cc8b-4706-a815-25372d7f87e3 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:15.342710322 +0000 UTC m=+67.755724216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls") pod "dns-default-t9ptt" (UID: "5ef761fc-cc8b-4706-a815-25372d7f87e3") : secret "dns-default-metrics-tls" not found
Jul 06 23:56:59.422410 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.422376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:56:59.456071 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.456041    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m2sjv_f57ae867-009e-4fd5-a888-c66d631c614b/serve-healthcheck-canary/0.log"
Jul 06 23:56:59.745720 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.745620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:59.745889 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.745741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:56:59.745889 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.745778    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:59.745889 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.745853    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:56:59.745995 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.745859    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:03.745844668 +0000 UTC m=+56.158858554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:56:59.745995 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:56:59.745912    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:03.745900096 +0000 UTC m=+56.158913982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : configmap "kube-rbac-proxy" not found
Jul 06 23:56:59.985516 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:56:59.985484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:03.577287 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.577251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.577294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.577348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.577393    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.577473    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls podName:061a0d8e-ce66-4d25-b524-839070cbd53c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:19.577440125 +0000 UTC m=+71.990454010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-djml9" (UID: "061a0d8e-ce66-4d25-b524-839070cbd53c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.577479    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.577540    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:19.577520886 +0000 UTC m=+71.990534792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : configmap references non-existent config key: service-ca.crt
Jul 06 23:57:03.577784 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.577566    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs podName:e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:19.577554532 +0000 UTC m=+71.990568423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs") pod "router-default-68846f8cb5-nx52d" (UID: "e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726") : secret "router-metrics-certs-default" not found
Jul 06 23:57:03.678523 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.678490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:57:03.680814 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.680783    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/381a88ba-f473-4669-8f6a-6e67289d9c8c-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qs6g9\" (UID: \"381a88ba-f473-4669-8f6a-6e67289d9c8c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:57:03.779662 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.779631    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:57:03.779776 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.779696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:57:03.779832 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.779817    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:03.779903 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:03.779893    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls podName:4e1ba836-686a-46b1-94e4-b5a4fab82637 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:11.779877472 +0000 UTC m=+64.192891357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r875p" (UID: "4e1ba836-686a-46b1-94e4-b5a4fab82637") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:03.780176 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.780159    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e1ba836-686a-46b1-94e4-b5a4fab82637-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:57:03.819498 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.819477    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"
Jul 06 23:57:03.940921 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:03.940785    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9"]
Jul 06 23:57:04.457043 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:04.457010    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9" event={"ID":"381a88ba-f473-4669-8f6a-6e67289d9c8c","Type":"ContainerStarted","Data":"36dcc44b9a13e6294b28c534ced40691a00f4519f78866e50f7a3535123c1ad0"}
Jul 06 23:57:06.464557 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:06.464523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9" event={"ID":"381a88ba-f473-4669-8f6a-6e67289d9c8c","Type":"ContainerStarted","Data":"c34b7f7b38d876567feb887412d377692af4ca4a6c9b3b6b9a84497336a3bd21"}
Jul 06 23:57:06.464983 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:06.464576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9" event={"ID":"381a88ba-f473-4669-8f6a-6e67289d9c8c","Type":"ContainerStarted","Data":"1d9fa6f8adf263c1af095e46ddfc36a89ed84ed3c7d9cab3f1518adb03fcccf1"}
Jul 06 23:57:06.482307 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:06.482260    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qs6g9" podStartSLOduration=18.029909009 podStartE2EDuration="19.482244605s" totalImagesPullingTime="1.452335596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:06.480542858 +0000 UTC m=+58.893556779" watchObservedRunningTime="2026-07-06 23:57:06.482244605 +0000 UTC m=+58.895258514"
Jul 06 23:57:10.365237 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:10.365211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xb55m"
Jul 06 23:57:11.109038 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.109007    2564 scope.go:117] "RemoveContainer" containerID="d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b"
Jul 06 23:57:11.481087 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.481016    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 06 23:57:11.481434 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.481373    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/1.log"
Jul 06 23:57:11.481434 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.481407    2564 generic.go:356] "Generic (PLEG): container finished" podID="541e45ab-dde2-48dc-b018-b187bcf1387a" containerID="234b36f280cf96cced3262bec3a3c770a8e6e270a0db9ccebe754b3e3b7ff31e" exitCode=255
Jul 06 23:57:11.481543 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.481488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" event={"ID":"541e45ab-dde2-48dc-b018-b187bcf1387a","Type":"ContainerDied","Data":"234b36f280cf96cced3262bec3a3c770a8e6e270a0db9ccebe754b3e3b7ff31e"}
Jul 06 23:57:11.481587 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.481546    2564 scope.go:117] "RemoveContainer" containerID="d9e8a9c98e91802792abdb0af37a5d1c3f6f2d58db48682aab441dcd9416e37b"
Jul 06 23:57:11.481872 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.481853    2564 scope.go:117] "RemoveContainer" containerID="234b36f280cf96cced3262bec3a3c770a8e6e270a0db9ccebe754b3e3b7ff31e"
Jul 06 23:57:11.482043 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:11.482025    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-cht2h_openshift-console-operator(541e45ab-dde2-48dc-b018-b187bcf1387a)\"" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podUID="541e45ab-dde2-48dc-b018-b187bcf1387a"
Jul 06 23:57:11.847929 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.847903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:57:11.850175 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.850145    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba836-686a-46b1-94e4-b5a4fab82637-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r875p\" (UID: \"4e1ba836-686a-46b1-94e4-b5a4fab82637\") " pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:57:11.931146 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.931121    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q9bzd\""
Jul 06 23:57:11.939124 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:11.939098    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r875p"
Jul 06 23:57:12.059085 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:12.059055    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r875p"]
Jul 06 23:57:12.061837 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:12.061812    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1ba836_686a_46b1_94e4_b5a4fab82637.slice/crio-542431605a91a4e5f792438f018f243aca367c6d4553987356f6da9e2e2f5fcd WatchSource:0}: Error finding container 542431605a91a4e5f792438f018f243aca367c6d4553987356f6da9e2e2f5fcd: Status 404 returned error can't find the container with id 542431605a91a4e5f792438f018f243aca367c6d4553987356f6da9e2e2f5fcd
Jul 06 23:57:12.489317 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:12.489284    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 06 23:57:12.491355 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:12.491280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r875p" event={"ID":"4e1ba836-686a-46b1-94e4-b5a4fab82637","Type":"ContainerStarted","Data":"542431605a91a4e5f792438f018f243aca367c6d4553987356f6da9e2e2f5fcd"}
Jul 06 23:57:13.865426 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.865393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:57:13.868293 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.868260    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:57:13.878151 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.878126    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1df4ec18-0a1d-4039-a90d-e7afc5fca917-metrics-certs\") pod \"network-metrics-daemon-wt5k2\" (UID: \"1df4ec18-0a1d-4039-a90d-e7afc5fca917\") " pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:57:13.926757 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.926718    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-knf8z\""
Jul 06 23:57:13.933649 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.933628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wt5k2"
Jul 06 23:57:13.966879 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.966848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:57:13.967024 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.967010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:57:13.969806 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.969785    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4wt\" (UniqueName: \"kubernetes.io/projected/4965cddf-5ea8-4899-9bb6-dc5032852c22-kube-api-access-4t4wt\") pod \"network-check-target-ccgjm\" (UID: \"4965cddf-5ea8-4899-9bb6-dc5032852c22\") " pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:57:13.969925 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.969844    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:57:13.979644 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:13.979613    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e044b7a4-d4c4-444c-a225-ffc0f2b5eb15-original-pull-secret\") pod \"global-pull-secret-syncer-5rgcf\" (UID: \"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15\") " pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:57:14.222065 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:14.221997    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lswch\""
Jul 06 23:57:14.227958 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:14.227932    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5rgcf"
Jul 06 23:57:14.228985 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:14.228958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:57:15.361527 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.361496    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ccgjm"]
Jul 06 23:57:15.377970 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.377942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:57:15.378086 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.378012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:57:15.380560 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.380536    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef761fc-cc8b-4706-a815-25372d7f87e3-metrics-tls\") pod \"dns-default-t9ptt\" (UID: \"5ef761fc-cc8b-4706-a815-25372d7f87e3\") " pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:57:15.380655 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.380594    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"image-registry-7b8fdf9db-9cqph\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") " pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:57:15.515322 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.515243    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hbkmd\""
Jul 06 23:57:15.522315 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.522277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:57:15.531034 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.531011    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jf85g\""
Jul 06 23:57:15.538590 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.538569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:57:15.581282 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:15.581247    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4965cddf_5ea8_4899_9bb6_dc5032852c22.slice/crio-2caf1e936c134f70964ea52e48d78c09426c7bceb670b9e659810943a6b14913 WatchSource:0}: Error finding container 2caf1e936c134f70964ea52e48d78c09426c7bceb670b9e659810943a6b14913: Status 404 returned error can't find the container with id 2caf1e936c134f70964ea52e48d78c09426c7bceb670b9e659810943a6b14913
Jul 06 23:57:15.746177 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.746151    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5rgcf"]
Jul 06 23:57:15.749782 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:15.749755    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode044b7a4_d4c4_444c_a225_ffc0f2b5eb15.slice/crio-651ef0f6536749f1a5a61aa8b33fe6664eab97252073a316f073b756a6e3ab0e WatchSource:0}: Error finding container 651ef0f6536749f1a5a61aa8b33fe6664eab97252073a316f073b756a6e3ab0e: Status 404 returned error can't find the container with id 651ef0f6536749f1a5a61aa8b33fe6664eab97252073a316f073b756a6e3ab0e
Jul 06 23:57:15.802398 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.802374    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b8fdf9db-9cqph"]
Jul 06 23:57:15.804581 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:15.804558    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28759ab_2f45_43d5_a277_1e74916bd272.slice/crio-5293e65cbd94e933b6b4cdf680612feb4e31e90a9b83718419bde83344431241 WatchSource:0}: Error finding container 5293e65cbd94e933b6b4cdf680612feb4e31e90a9b83718419bde83344431241: Status 404 returned error can't find the container with id 5293e65cbd94e933b6b4cdf680612feb4e31e90a9b83718419bde83344431241
Jul 06 23:57:15.965675 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.965543    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t9ptt"]
Jul 06 23:57:15.966573 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:15.966553    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wt5k2"]
Jul 06 23:57:15.968128 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:15.968103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef761fc_cc8b_4706_a815_25372d7f87e3.slice/crio-5f42ce7076cb80c31d669ab1c43ec6f15bdccdfb9f8c2ef4ff4e8247fe99a57d WatchSource:0}: Error finding container 5f42ce7076cb80c31d669ab1c43ec6f15bdccdfb9f8c2ef4ff4e8247fe99a57d: Status 404 returned error can't find the container with id 5f42ce7076cb80c31d669ab1c43ec6f15bdccdfb9f8c2ef4ff4e8247fe99a57d
Jul 06 23:57:15.968842 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:15.968818    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df4ec18_0a1d_4039_a90d_e7afc5fca917.slice/crio-3e488ea76dfc595120ca481c18e65a01640c06f6f5a8c8a36cd1dc97d3163f8f WatchSource:0}: Error finding container 3e488ea76dfc595120ca481c18e65a01640c06f6f5a8c8a36cd1dc97d3163f8f: Status 404 returned error can't find the container with id 3e488ea76dfc595120ca481c18e65a01640c06f6f5a8c8a36cd1dc97d3163f8f
Jul 06 23:57:16.506997 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.506953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9ptt" event={"ID":"5ef761fc-cc8b-4706-a815-25372d7f87e3","Type":"ContainerStarted","Data":"5f42ce7076cb80c31d669ab1c43ec6f15bdccdfb9f8c2ef4ff4e8247fe99a57d"}
Jul 06 23:57:16.508264 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.508231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wt5k2" event={"ID":"1df4ec18-0a1d-4039-a90d-e7afc5fca917","Type":"ContainerStarted","Data":"3e488ea76dfc595120ca481c18e65a01640c06f6f5a8c8a36cd1dc97d3163f8f"}
Jul 06 23:57:16.509513 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.509484    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5rgcf" event={"ID":"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15","Type":"ContainerStarted","Data":"651ef0f6536749f1a5a61aa8b33fe6664eab97252073a316f073b756a6e3ab0e"}
Jul 06 23:57:16.511120 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.511099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r875p" event={"ID":"4e1ba836-686a-46b1-94e4-b5a4fab82637","Type":"ContainerStarted","Data":"eacc49c3ac9d2a05244f8b65f7de102346f3a43c56ad609e8da685276248e982"}
Jul 06 23:57:16.512662 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.512636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ccgjm" event={"ID":"4965cddf-5ea8-4899-9bb6-dc5032852c22","Type":"ContainerStarted","Data":"54b57175c7013d87d28e5e96054016eb14cf9838720a1ab3d06c77e3cc8d4b28"}
Jul 06 23:57:16.512789 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.512688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ccgjm" event={"ID":"4965cddf-5ea8-4899-9bb6-dc5032852c22","Type":"ContainerStarted","Data":"2caf1e936c134f70964ea52e48d78c09426c7bceb670b9e659810943a6b14913"}
Jul 06 23:57:16.512789 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.512713    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:57:16.514325 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.514298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph" event={"ID":"c28759ab-2f45-43d5-a277-1e74916bd272","Type":"ContainerStarted","Data":"5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0"}
Jul 06 23:57:16.514448 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.514342    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph" event={"ID":"c28759ab-2f45-43d5-a277-1e74916bd272","Type":"ContainerStarted","Data":"5293e65cbd94e933b6b4cdf680612feb4e31e90a9b83718419bde83344431241"}
Jul 06 23:57:16.514568 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.514449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:57:16.584507 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:16.584442    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ccgjm" podStartSLOduration=67.584428574 podStartE2EDuration="1m7.584428574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:16.547583385 +0000 UTC m=+68.960597312" watchObservedRunningTime="2026-07-06 23:57:16.584428574 +0000 UTC m=+68.997442504"
Jul 06 23:57:17.521955 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:17.521916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r875p" event={"ID":"4e1ba836-686a-46b1-94e4-b5a4fab82637","Type":"ContainerStarted","Data":"57ba91abe17883feff8989a7bf68c4c6acc7e596ae879b3778f9b0fa5a6d53f7"}
Jul 06 23:57:18.132733 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:18.132613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:57:18.132733 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:18.132671    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:57:18.133012 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:18.132993    2564 scope.go:117] "RemoveContainer" containerID="234b36f280cf96cced3262bec3a3c770a8e6e270a0db9ccebe754b3e3b7ff31e"
Jul 06 23:57:18.133540 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:18.133187    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-cht2h_openshift-console-operator(541e45ab-dde2-48dc-b018-b187bcf1387a)\"" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podUID="541e45ab-dde2-48dc-b018-b187bcf1387a"
Jul 06 23:57:18.164027 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:18.163936    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph" podStartSLOduration=69.16391523 podStartE2EDuration="1m9.16391523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:16.591685056 +0000 UTC m=+69.004698965" watchObservedRunningTime="2026-07-06 23:57:18.16391523 +0000 UTC m=+70.576929137"
Jul 06 23:57:18.527433 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:18.527395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wt5k2" event={"ID":"1df4ec18-0a1d-4039-a90d-e7afc5fca917","Type":"ContainerStarted","Data":"9518d9a942fa9fb8943349d2fd2bfbbcfdb7b67d7feb425c381d451b02e35891"}
Jul 06 23:57:19.617829 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.617793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:19.618290 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.617866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:19.618290 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.617941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:57:19.618624 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.618600    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-service-ca-bundle\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:19.620442 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.620418    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/061a0d8e-ce66-4d25-b524-839070cbd53c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-djml9\" (UID: \"061a0d8e-ce66-4d25-b524-839070cbd53c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:57:19.620575 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.620481    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726-metrics-certs\") pod \"router-default-68846f8cb5-nx52d\" (UID: \"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726\") " pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:19.887755 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.887676    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zckl5\""
Jul 06 23:57:19.895514 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.895489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"
Jul 06 23:57:19.900798 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.900780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-njrfs\""
Jul 06 23:57:19.908084 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:19.908058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:20.925215 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.925182    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"]
Jul 06 23:57:20.938627 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.938605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:20.941683 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.941659    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 23:57:20.941813 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.941663    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 23:57:20.942863 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.942844    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 23:57:20.942989 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.942856    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 23:57:20.942989 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.942925    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 23:57:20.943750 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.943626    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 23:57:20.948673 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.948654    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 23:57:20.967836 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:20.967815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"]
Jul 06 23:57:21.130215 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.130190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-ca\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.130337 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.130241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.130337 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.130302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x4tx\" (UniqueName: \"kubernetes.io/projected/b162ee25-573f-4a19-8b18-2114c737f5e2-kube-api-access-6x4tx\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.130411 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.130376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.130492 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.130414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-hub\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.130492 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.130465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b162ee25-573f-4a19-8b18-2114c737f5e2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.233303 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.231921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.233303 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.231975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x4tx\" (UniqueName: \"kubernetes.io/projected/b162ee25-573f-4a19-8b18-2114c737f5e2-kube-api-access-6x4tx\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.233303 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.232033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.233303 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.232069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-hub\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.233303 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.232107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b162ee25-573f-4a19-8b18-2114c737f5e2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.233303 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.232138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-ca\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.234694 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.234647    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b162ee25-573f-4a19-8b18-2114c737f5e2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.237721 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.237697    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-ca\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.253110 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.252993    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-hub\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.254208 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.254181    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.259913 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.259867    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b162ee25-573f-4a19-8b18-2114c737f5e2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.265212 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.265185    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x4tx\" (UniqueName: \"kubernetes.io/projected/b162ee25-573f-4a19-8b18-2114c737f5e2-kube-api-access-6x4tx\") pod \"cluster-proxy-proxy-agent-5bb4499d96-k9rpb\" (UID: \"b162ee25-573f-4a19-8b18-2114c737f5e2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.329107 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.329036    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68846f8cb5-nx52d"]
Jul 06 23:57:21.334228 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:21.334202    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53b48d6_7cc9_4fdf_b6ea_5c5463ce4726.slice/crio-13d5eb589aa43497339d6454681ae3263668cc2252f6862d79c2a606d0ac06ea WatchSource:0}: Error finding container 13d5eb589aa43497339d6454681ae3263668cc2252f6862d79c2a606d0ac06ea: Status 404 returned error can't find the container with id 13d5eb589aa43497339d6454681ae3263668cc2252f6862d79c2a606d0ac06ea
Jul 06 23:57:21.349178 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.347595    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9"]
Jul 06 23:57:21.540203 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.540170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9ptt" event={"ID":"5ef761fc-cc8b-4706-a815-25372d7f87e3","Type":"ContainerStarted","Data":"ae74b1624138d15cec4371f008a232443a49f73bfafcc61065087a28930205a7"}
Jul 06 23:57:21.540369 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.540223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t9ptt" event={"ID":"5ef761fc-cc8b-4706-a815-25372d7f87e3","Type":"ContainerStarted","Data":"1a4e759f1bcd6b102b99e0b3f6a02b43bfe7255aab071c9001834e5207314512"}
Jul 06 23:57:21.540369 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.540323    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:57:21.541913 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.541887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wt5k2" event={"ID":"1df4ec18-0a1d-4039-a90d-e7afc5fca917","Type":"ContainerStarted","Data":"b1bb29900e6651f8a56bd7a4d378979935a5ee6500d417be22a56d44e874d991"}
Jul 06 23:57:21.543374 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.543352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5rgcf" event={"ID":"e044b7a4-d4c4-444c-a225-ffc0f2b5eb15","Type":"ContainerStarted","Data":"e00079000b68053e82766e1f021d11c149b29ba57b368ce39aa3ae9f2674f1ee"}
Jul 06 23:57:21.544822 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.544790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68846f8cb5-nx52d" event={"ID":"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726","Type":"ContainerStarted","Data":"07ee4792939789e80455a94a87e1164dcddde5f8b40bbc22480a1a7b23752397"}
Jul 06 23:57:21.544929 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.544832    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68846f8cb5-nx52d" event={"ID":"e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726","Type":"ContainerStarted","Data":"13d5eb589aa43497339d6454681ae3263668cc2252f6862d79c2a606d0ac06ea"}
Jul 06 23:57:21.546670 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.546647    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r875p" event={"ID":"4e1ba836-686a-46b1-94e4-b5a4fab82637","Type":"ContainerStarted","Data":"9f7049e774f6d5eae005a7d234b21a7823f0ca4c8a941b5d9b10212f76d9cbb2"}
Jul 06 23:57:21.547739 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.547721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9" event={"ID":"061a0d8e-ce66-4d25-b524-839070cbd53c","Type":"ContainerStarted","Data":"2492411e4aaa7313ef47b569bc99c337e9b60699deae269d9ce1adcc8560e938"}
Jul 06 23:57:21.557813 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.557790    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"
Jul 06 23:57:21.568955 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.568916    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t9ptt" podStartSLOduration=33.397714127 podStartE2EDuration="38.568903026s" totalImagesPullingTime="5.171188899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:21.568097508 +0000 UTC m=+73.981111417" watchObservedRunningTime="2026-07-06 23:57:21.568903026 +0000 UTC m=+73.981916934"
Jul 06 23:57:21.602127 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.602063    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wt5k2" podStartSLOduration=70.228138486 podStartE2EDuration="1m12.602039269s" totalImagesPullingTime="2.373900783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:21.600905175 +0000 UTC m=+74.013919084" watchObservedRunningTime="2026-07-06 23:57:21.602039269 +0000 UTC m=+74.015053245"
Jul 06 23:57:21.639930 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.639879    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68846f8cb5-nx52d" podStartSLOduration=34.639862388 podStartE2EDuration="34.639862388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:21.638128919 +0000 UTC m=+74.051142827" watchObservedRunningTime="2026-07-06 23:57:21.639862388 +0000 UTC m=+74.052876290"
Jul 06 23:57:21.666805 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.666756    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r875p" podStartSLOduration=17.799332471 podStartE2EDuration="26.666738497s" totalImagesPullingTime="8.867406026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:21.66479059 +0000 UTC m=+74.077804517" watchObservedRunningTime="2026-07-06 23:57:21.666738497 +0000 UTC m=+74.079752405"
Jul 06 23:57:21.689341 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.689255    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5rgcf" podStartSLOduration=67.291140402 podStartE2EDuration="1m12.689242236s" totalImagesPullingTime="5.398101834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:21.688802674 +0000 UTC m=+74.101816612" watchObservedRunningTime="2026-07-06 23:57:21.689242236 +0000 UTC m=+74.102256145"
Jul 06 23:57:21.700167 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.700141    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb"]
Jul 06 23:57:21.702976 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:21.702940    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb162ee25_573f_4a19_8b18_2114c737f5e2.slice/crio-816d1e0f4aaba6b6169f59dee6f9f475fda7b9fbf5fc84580cef69eed9bd7fd8 WatchSource:0}: Error finding container 816d1e0f4aaba6b6169f59dee6f9f475fda7b9fbf5fc84580cef69eed9bd7fd8: Status 404 returned error can't find the container with id 816d1e0f4aaba6b6169f59dee6f9f475fda7b9fbf5fc84580cef69eed9bd7fd8
Jul 06 23:57:21.908582 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.908547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:21.911613 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:21.911578    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:22.554403 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:22.554360    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb" event={"ID":"b162ee25-573f-4a19-8b18-2114c737f5e2","Type":"ContainerStarted","Data":"816d1e0f4aaba6b6169f59dee6f9f475fda7b9fbf5fc84580cef69eed9bd7fd8"}
Jul 06 23:57:22.556354 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:22.556295    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:22.558381 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:22.558111    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68846f8cb5-nx52d"
Jul 06 23:57:24.563101 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:24.563061    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9" event={"ID":"061a0d8e-ce66-4d25-b524-839070cbd53c","Type":"ContainerStarted","Data":"ba87565a5ff534633af66704abcd35468c7ad30c1f9a325be1f92ec471a2c85f"}
Jul 06 23:57:24.587880 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:24.587833    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-djml9" podStartSLOduration=35.071887789 podStartE2EDuration="37.587802982s" totalImagesPullingTime="2.515915193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:24.585844132 +0000 UTC m=+76.998858058" watchObservedRunningTime="2026-07-06 23:57:24.587802982 +0000 UTC m=+77.000816890"
Jul 06 23:57:26.572380 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:26.572300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb" event={"ID":"b162ee25-573f-4a19-8b18-2114c737f5e2","Type":"ContainerStarted","Data":"f56b6e3244b66b2c1d1b10f6e7ebe342fd56ba900586faea3d09328cbed075f2"}
Jul 06 23:57:26.572380 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:26.572342    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb" event={"ID":"b162ee25-573f-4a19-8b18-2114c737f5e2","Type":"ContainerStarted","Data":"8e133c12efe6b9013bfcaf0e502512fac188d25e7abc26b9547f7aa6df3b4fc4"}
Jul 06 23:57:26.572380 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:26.572360    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb" event={"ID":"b162ee25-573f-4a19-8b18-2114c737f5e2","Type":"ContainerStarted","Data":"fca80a29df1d3d179ffa50d63d12cd1786d508d2b876eb6ed352ad991c7650d1"}
Jul 06 23:57:26.602598 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:26.602530    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb4499d96-k9rpb" podStartSLOduration=2.382809975 podStartE2EDuration="6.602515321s" totalImagesPullingTime="4.219705346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:26.598894019 +0000 UTC m=+79.011907940" watchObservedRunningTime="2026-07-06 23:57:26.602515321 +0000 UTC m=+79.015529235"
Jul 06 23:57:31.109526 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:31.109368    2564 scope.go:117] "RemoveContainer" containerID="234b36f280cf96cced3262bec3a3c770a8e6e270a0db9ccebe754b3e3b7ff31e"
Jul 06 23:57:31.110041 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:57:31.109583    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-cht2h_openshift-console-operator(541e45ab-dde2-48dc-b018-b187bcf1387a)\"" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podUID="541e45ab-dde2-48dc-b018-b187bcf1387a"
Jul 06 23:57:31.557369 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:31.557288    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t9ptt"
Jul 06 23:57:37.046420 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.046382    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g7j7q"]
Jul 06 23:57:37.050731 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.050710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.053313 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.053291    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 23:57:37.053448 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.053381    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f7qp4\""
Jul 06 23:57:37.053743 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.053725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 23:57:37.054626 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.054611    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 23:57:37.054682 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.054643    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 23:57:37.152101 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152058    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-accelerators-collector-config\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152276 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-wtmp\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152276 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152150    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-sys\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152276 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9spg\" (UniqueName: \"kubernetes.io/projected/37c86b59-d35e-4a6c-89c0-d29199c02372-kube-api-access-g9spg\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152276 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-tls\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152276 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37c86b59-d35e-4a6c-89c0-d29199c02372-metrics-client-ca\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152543 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152299    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152543 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152350    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-root\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.152543 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.152386    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-textfile\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.252964 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.252926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253139 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.252982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-root\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253139 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-textfile\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253139 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-accelerators-collector-config\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253139 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-wtmp\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-sys\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9spg\" (UniqueName: \"kubernetes.io/projected/37c86b59-d35e-4a6c-89c0-d29199c02372-kube-api-access-g9spg\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253244    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-wtmp\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-tls\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37c86b59-d35e-4a6c-89c0-d29199c02372-metrics-client-ca\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253294    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-sys\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253364 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253116    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/37c86b59-d35e-4a6c-89c0-d29199c02372-root\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253768 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253736    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-textfile\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.253924 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.253896    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37c86b59-d35e-4a6c-89c0-d29199c02372-metrics-client-ca\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.254331 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.254309    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-accelerators-collector-config\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.255974 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.255946    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.256090 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.256052    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/37c86b59-d35e-4a6c-89c0-d29199c02372-node-exporter-tls\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.265322 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.265292    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9spg\" (UniqueName: \"kubernetes.io/projected/37c86b59-d35e-4a6c-89c0-d29199c02372-kube-api-access-g9spg\") pod \"node-exporter-g7j7q\" (UID: \"37c86b59-d35e-4a6c-89c0-d29199c02372\") " pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.364086 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.364062    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g7j7q"
Jul 06 23:57:37.375597 ip-10-0-141-123 kubenswrapper[2564]: W0706 23:57:37.375572    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c86b59_d35e_4a6c_89c0_d29199c02372.slice/crio-a26e669e527d089e1aa4302506a653248becf0ea537f75f07193945c0749c21d WatchSource:0}: Error finding container a26e669e527d089e1aa4302506a653248becf0ea537f75f07193945c0749c21d: Status 404 returned error can't find the container with id a26e669e527d089e1aa4302506a653248becf0ea537f75f07193945c0749c21d
Jul 06 23:57:37.526225 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.526198    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:57:37.614166 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:37.614081    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g7j7q" event={"ID":"37c86b59-d35e-4a6c-89c0-d29199c02372","Type":"ContainerStarted","Data":"a26e669e527d089e1aa4302506a653248becf0ea537f75f07193945c0749c21d"}
Jul 06 23:57:38.619493 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:38.619427    2564 generic.go:356] "Generic (PLEG): container finished" podID="37c86b59-d35e-4a6c-89c0-d29199c02372" containerID="762a6614cadef0ff4a326588c2b6a1c7f421d65afba2341a9a9c47675b98ece6" exitCode=0
Jul 06 23:57:38.619871 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:38.619522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g7j7q" event={"ID":"37c86b59-d35e-4a6c-89c0-d29199c02372","Type":"ContainerDied","Data":"762a6614cadef0ff4a326588c2b6a1c7f421d65afba2341a9a9c47675b98ece6"}
Jul 06 23:57:39.629614 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:39.629578    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g7j7q" event={"ID":"37c86b59-d35e-4a6c-89c0-d29199c02372","Type":"ContainerStarted","Data":"3824f713ea16457d66fc87053fc21e45ccf1d33c1ff73f561b780a3c3648e341"}
Jul 06 23:57:39.629986 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:39.629651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g7j7q" event={"ID":"37c86b59-d35e-4a6c-89c0-d29199c02372","Type":"ContainerStarted","Data":"a6dc07d1041c1c048fd6964e62b67447b1d16350924b1c62f802d9ed71c7a461"}
Jul 06 23:57:39.650702 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:39.650654    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g7j7q" podStartSLOduration=1.691895357 podStartE2EDuration="2.650639281s" totalImagesPullingTime="895.230933ms" totalInitContainerRuntime="63.512991ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:39.649128703 +0000 UTC m=+92.062142634" watchObservedRunningTime="2026-07-06 23:57:39.650639281 +0000 UTC m=+92.063653189"
Jul 06 23:57:43.010268 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:43.010228    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b8fdf9db-9cqph"]
Jul 06 23:57:46.109903 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:46.109874    2564 scope.go:117] "RemoveContainer" containerID="234b36f280cf96cced3262bec3a3c770a8e6e270a0db9ccebe754b3e3b7ff31e"
Jul 06 23:57:46.657274 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:46.657242    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 06 23:57:46.657475 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:46.657338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" event={"ID":"541e45ab-dde2-48dc-b018-b187bcf1387a","Type":"ContainerStarted","Data":"420e0b7547714696834dfe45f19f09fc9d5b49265c7154eebc71e91d764bf5c0"}
Jul 06 23:57:46.657713 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:46.657690    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:57:46.674251 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:46.674199    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-cht2h" podStartSLOduration=53.736381003 podStartE2EDuration="59.674187024s" totalImagesPullingTime="5.937806021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:46.673619237 +0000 UTC m=+99.086633148" watchObservedRunningTime="2026-07-06 23:57:46.674187024 +0000 UTC m=+99.087200956"
Jul 06 23:57:47.483347 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:47.483319    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-cht2h"
Jul 06 23:57:47.525320 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:57:47.525291    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ccgjm"
Jul 06 23:58:08.037263 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.037221    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph" podUID="c28759ab-2f45-43d5-a277-1e74916bd272" containerName="registry" containerID="cri-o://5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0" gracePeriod=30
Jul 06 23:58:08.278747 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.278721    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:58:08.403096 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403067    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jghjh\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-kube-api-access-jghjh\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403280 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403114    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-image-registry-private-configuration\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403280 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403150    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-registry-certificates\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403280 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-trusted-ca\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403280 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c28759ab-2f45-43d5-a277-1e74916bd272-ca-trust-extracted\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403280 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403227    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-bound-sa-token\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403280 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403255    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-installation-pull-secrets\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.403606 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403304    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") pod \"c28759ab-2f45-43d5-a277-1e74916bd272\" (UID: \"c28759ab-2f45-43d5-a277-1e74916bd272\") "
Jul 06 23:58:08.404019 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403954    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:08.404019 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.403964    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:58:08.406083 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.406024    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:08.406083 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.406035    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:08.406218 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.406062    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:08.406218 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.406143    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:58:08.406289 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.406226    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-kube-api-access-jghjh" (OuterVolumeSpecName: "kube-api-access-jghjh") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "kube-api-access-jghjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:58:08.411795 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.411772    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c28759ab-2f45-43d5-a277-1e74916bd272-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c28759ab-2f45-43d5-a277-1e74916bd272" (UID: "c28759ab-2f45-43d5-a277-1e74916bd272"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:58:08.504545 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504512    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-bound-sa-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504545 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504539    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-installation-pull-secrets\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504545 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504551    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-registry-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504750 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504559    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jghjh\" (UniqueName: \"kubernetes.io/projected/c28759ab-2f45-43d5-a277-1e74916bd272-kube-api-access-jghjh\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504750 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504569    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c28759ab-2f45-43d5-a277-1e74916bd272-image-registry-private-configuration\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504750 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504578    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-registry-certificates\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504750 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504586    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c28759ab-2f45-43d5-a277-1e74916bd272-trusted-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.504750 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.504594    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c28759ab-2f45-43d5-a277-1e74916bd272-ca-trust-extracted\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 06 23:58:08.743814 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.743717    2564 generic.go:356] "Generic (PLEG): container finished" podID="c28759ab-2f45-43d5-a277-1e74916bd272" containerID="5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0" exitCode=0
Jul 06 23:58:08.743959 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.743824    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph"
Jul 06 23:58:08.743959 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.743834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph" event={"ID":"c28759ab-2f45-43d5-a277-1e74916bd272","Type":"ContainerDied","Data":"5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0"}
Jul 06 23:58:08.743959 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.743892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b8fdf9db-9cqph" event={"ID":"c28759ab-2f45-43d5-a277-1e74916bd272","Type":"ContainerDied","Data":"5293e65cbd94e933b6b4cdf680612feb4e31e90a9b83718419bde83344431241"}
Jul 06 23:58:08.743959 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.743913    2564 scope.go:117] "RemoveContainer" containerID="5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0"
Jul 06 23:58:08.752311 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.752293    2564 scope.go:117] "RemoveContainer" containerID="5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0"
Jul 06 23:58:08.752619 ip-10-0-141-123 kubenswrapper[2564]: E0706 23:58:08.752598    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0\": container with ID starting with 5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0 not found: ID does not exist" containerID="5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0"
Jul 06 23:58:08.752696 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.752628    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0"} err="failed to get container status \"5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0\": rpc error: code = NotFound desc = could not find container \"5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0\": container with ID starting with 5a0707e53cde12c17e32c12e813ad7af6349a452235c9061d65474377eb68ac0 not found: ID does not exist"
Jul 06 23:58:08.765327 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.765306    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b8fdf9db-9cqph"]
Jul 06 23:58:08.769854 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:08.769834    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b8fdf9db-9cqph"]
Jul 06 23:58:10.113272 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:10.113231    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c28759ab-2f45-43d5-a277-1e74916bd272" path="/var/lib/kubelet/pods/c28759ab-2f45-43d5-a277-1e74916bd272/volumes"
Jul 06 23:58:15.770408 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:15.770369    2564 generic.go:356] "Generic (PLEG): container finished" podID="7176dbf9-7969-411f-9efb-dec85745eed8" containerID="c5d60fb3976735bfa0dec394ae264acf305ba6d1a2f9ef149a53caa2483637bb" exitCode=0
Jul 06 23:58:15.770849 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:15.770439    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm" event={"ID":"7176dbf9-7969-411f-9efb-dec85745eed8","Type":"ContainerDied","Data":"c5d60fb3976735bfa0dec394ae264acf305ba6d1a2f9ef149a53caa2483637bb"}
Jul 06 23:58:15.770849 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:15.770774    2564 scope.go:117] "RemoveContainer" containerID="c5d60fb3976735bfa0dec394ae264acf305ba6d1a2f9ef149a53caa2483637bb"
Jul 06 23:58:16.775686 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:16.775652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dmldm" event={"ID":"7176dbf9-7969-411f-9efb-dec85745eed8","Type":"ContainerStarted","Data":"077eec6bb724e83fc765510f3f258c6a2458fbb8866dcfc4cfadb5aa64156fab"}
Jul 06 23:58:20.790321 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:20.790283    2564 generic.go:356] "Generic (PLEG): container finished" podID="637b2feb-ed55-4e15-889e-9d8eb9becc51" containerID="047b5787f29554a21e002b080a4a9127718fec672f2b641cce0f5d12ea1e1b88" exitCode=0
Jul 06 23:58:20.790729 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:20.790353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns" event={"ID":"637b2feb-ed55-4e15-889e-9d8eb9becc51","Type":"ContainerDied","Data":"047b5787f29554a21e002b080a4a9127718fec672f2b641cce0f5d12ea1e1b88"}
Jul 06 23:58:20.790729 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:20.790691    2564 scope.go:117] "RemoveContainer" containerID="047b5787f29554a21e002b080a4a9127718fec672f2b641cce0f5d12ea1e1b88"
Jul 06 23:58:21.795574 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:21.795533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-rppns" event={"ID":"637b2feb-ed55-4e15-889e-9d8eb9becc51","Type":"ContainerStarted","Data":"f606885ee6ad9b105922ea1d4c65d69b41e7dc42d9141993727e7ed008259f21"}
Jul 06 23:58:25.811162 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:25.811078    2564 generic.go:356] "Generic (PLEG): container finished" podID="07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc" containerID="a9af6b516cb9ec4fbf15090f79e5b1595108ca2255ab22915730874079f45d1b" exitCode=0
Jul 06 23:58:25.811162 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:25.811111    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q" event={"ID":"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc","Type":"ContainerDied","Data":"a9af6b516cb9ec4fbf15090f79e5b1595108ca2255ab22915730874079f45d1b"}
Jul 06 23:58:25.811640 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:25.811433    2564 scope.go:117] "RemoveContainer" containerID="a9af6b516cb9ec4fbf15090f79e5b1595108ca2255ab22915730874079f45d1b"
Jul 06 23:58:26.815848 ip-10-0-141-123 kubenswrapper[2564]: I0706 23:58:26.815816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-hkx2q" event={"ID":"07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc","Type":"ContainerStarted","Data":"0e3ad51b709169a9e0a9918d08dd92b7583aefc2d0974a299c747801b9c489f1"}
Jul 07 00:01:08.005300 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:08.005270    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:01:08.016308 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:08.016284    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 00:01:08.017327 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:08.017306    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:01:26.755788 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.755752    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"]
Jul 07 00:01:26.758183 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.756045    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c28759ab-2f45-43d5-a277-1e74916bd272" containerName="registry"
Jul 07 00:01:26.758183 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.756056    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c28759ab-2f45-43d5-a277-1e74916bd272" containerName="registry"
Jul 07 00:01:26.758183 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.756112    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c28759ab-2f45-43d5-a277-1e74916bd272" containerName="registry"
Jul 07 00:01:26.759000 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.758984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.763361 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.763339    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 00:01:26.763822 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.763806    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 00:01:26.764071 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.764045    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 00:01:26.764149 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.764055    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jl66k\""
Jul 07 00:01:26.764210 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.764179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 00:01:26.780529 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.780505    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"]
Jul 07 00:01:26.829354 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.829322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa93f0f7-0145-4ac0-9309-d9fb429247d1-manifests\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.829354 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.829357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa93f0f7-0145-4ac0-9309-d9fb429247d1-tmp\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.829585 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.829382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa93f0f7-0145-4ac0-9309-d9fb429247d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.829585 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.829448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa93f0f7-0145-4ac0-9309-d9fb429247d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.829585 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.829552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7vhx\" (UniqueName: \"kubernetes.io/projected/fa93f0f7-0145-4ac0-9309-d9fb429247d1-kube-api-access-c7vhx\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.930790 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.930747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa93f0f7-0145-4ac0-9309-d9fb429247d1-tmp\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.930790 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.930794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa93f0f7-0145-4ac0-9309-d9fb429247d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.931061 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.930915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa93f0f7-0145-4ac0-9309-d9fb429247d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.931061 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.930960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7vhx\" (UniqueName: \"kubernetes.io/projected/fa93f0f7-0145-4ac0-9309-d9fb429247d1-kube-api-access-c7vhx\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.931061 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.931009    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa93f0f7-0145-4ac0-9309-d9fb429247d1-manifests\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.931313 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.931283    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa93f0f7-0145-4ac0-9309-d9fb429247d1-manifests\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.933164 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.933143    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa93f0f7-0145-4ac0-9309-d9fb429247d1-tmp\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.933278 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.933229    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa93f0f7-0145-4ac0-9309-d9fb429247d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.933332 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.933273    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa93f0f7-0145-4ac0-9309-d9fb429247d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:26.950064 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:26.950037    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7vhx\" (UniqueName: \"kubernetes.io/projected/fa93f0f7-0145-4ac0-9309-d9fb429247d1-kube-api-access-c7vhx\") pod \"opendatahub-operator-controller-manager-fcdfc77c6-w8xms\" (UID: \"fa93f0f7-0145-4ac0-9309-d9fb429247d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:27.069584 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:27.069498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:27.204942 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:27.204915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"]
Jul 07 00:01:27.208898 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:27.208871    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa93f0f7_0145_4ac0_9309_d9fb429247d1.slice/crio-5d7655472ae49d9093a9efec919a5239a4736a2d1ebcd17254d342b227fd2070 WatchSource:0}: Error finding container 5d7655472ae49d9093a9efec919a5239a4736a2d1ebcd17254d342b227fd2070: Status 404 returned error can't find the container with id 5d7655472ae49d9093a9efec919a5239a4736a2d1ebcd17254d342b227fd2070
Jul 07 00:01:27.210579 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:27.210563    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:01:27.446340 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:27.446306    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms" event={"ID":"fa93f0f7-0145-4ac0-9309-d9fb429247d1","Type":"ContainerStarted","Data":"5d7655472ae49d9093a9efec919a5239a4736a2d1ebcd17254d342b227fd2070"}
Jul 07 00:01:31.464888 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:31.464842    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms" event={"ID":"fa93f0f7-0145-4ac0-9309-d9fb429247d1","Type":"ContainerStarted","Data":"89af79da5de6dee3dc903d2e1b5bf8248478a516da28def715b59d84e7343095"}
Jul 07 00:01:32.470128 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:32.470096    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa93f0f7-0145-4ac0-9309-d9fb429247d1" containerID="89af79da5de6dee3dc903d2e1b5bf8248478a516da28def715b59d84e7343095" exitCode=0
Jul 07 00:01:32.470551 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:32.470184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms" event={"ID":"fa93f0f7-0145-4ac0-9309-d9fb429247d1","Type":"ContainerDied","Data":"89af79da5de6dee3dc903d2e1b5bf8248478a516da28def715b59d84e7343095"}
Jul 07 00:01:33.475147 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:33.475114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms" event={"ID":"fa93f0f7-0145-4ac0-9309-d9fb429247d1","Type":"ContainerStarted","Data":"3758aed2dc928cb25212608f9f1f51c1d7c6d0602b2227b4b3c5ea66baa9bcd3"}
Jul 07 00:01:33.475632 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:33.475227    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:44.482083 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:44.482050    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms"
Jul 07 00:01:44.506014 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:44.505969    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fcdfc77c6-w8xms" podStartSLOduration=13.733494106 podStartE2EDuration="18.505953449s" totalImagesPullingTime="3.712959777s" totalInitContainerRuntime="1.059499566s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:33.538669167 +0000 UTC m=+325.951683077" watchObservedRunningTime="2026-07-07 00:01:44.505953449 +0000 UTC m=+336.918967358"
Jul 07 00:01:47.282077 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.281989    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"]
Jul 07 00:01:47.293723 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.293691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.301147 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.301122    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 00:01:47.301277 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.301227    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 00:01:47.382901 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.382870    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"]
Jul 07 00:01:47.389883 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.389858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdb0be87-65eb-40b6-9c56-e07ddee42b88-tmp\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.390017 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.389892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs7fq\" (UniqueName: \"kubernetes.io/projected/cdb0be87-65eb-40b6-9c56-e07ddee42b88-kube-api-access-xs7fq\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.390017 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.389916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb0be87-65eb-40b6-9c56-e07ddee42b88-tls-certs\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.490807 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.490768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdb0be87-65eb-40b6-9c56-e07ddee42b88-tmp\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.490977 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.490806    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs7fq\" (UniqueName: \"kubernetes.io/projected/cdb0be87-65eb-40b6-9c56-e07ddee42b88-kube-api-access-xs7fq\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.490977 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.490846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb0be87-65eb-40b6-9c56-e07ddee42b88-tls-certs\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.493099 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.493067    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdb0be87-65eb-40b6-9c56-e07ddee42b88-tmp\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.493280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.493261    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdb0be87-65eb-40b6-9c56-e07ddee42b88-tls-certs\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.505400 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.505375    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-f7xn7"]
Jul 07 00:01:47.508653 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.508636    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:47.511355 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.511335    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bdllg\""
Jul 07 00:01:47.511649 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.511634    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 00:01:47.525095 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.525072    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs7fq\" (UniqueName: \"kubernetes.io/projected/cdb0be87-65eb-40b6-9c56-e07ddee42b88-kube-api-access-xs7fq\") pod \"kube-auth-proxy-56d7555db7-x88l5\" (UID: \"cdb0be87-65eb-40b6-9c56-e07ddee42b88\") " pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.539320 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.539225    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-f7xn7"]
Jul 07 00:01:47.603340 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.603290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"
Jul 07 00:01:47.692042 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.691958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0250feb6-d7bc-429e-bde9-d16d3e95ed96-cert\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:47.692042 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.692004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8nsx\" (UniqueName: \"kubernetes.io/projected/0250feb6-d7bc-429e-bde9-d16d3e95ed96-kube-api-access-q8nsx\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:47.746123 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.746086    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56d7555db7-x88l5"]
Jul 07 00:01:47.750045 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:47.750021    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb0be87_65eb_40b6_9c56_e07ddee42b88.slice/crio-8ff9cadfcf29ceccfc3dc575be1dd1ee8101d5fa012509cb1d09804d6f7dfd98 WatchSource:0}: Error finding container 8ff9cadfcf29ceccfc3dc575be1dd1ee8101d5fa012509cb1d09804d6f7dfd98: Status 404 returned error can't find the container with id 8ff9cadfcf29ceccfc3dc575be1dd1ee8101d5fa012509cb1d09804d6f7dfd98
Jul 07 00:01:47.793563 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.793482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0250feb6-d7bc-429e-bde9-d16d3e95ed96-cert\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:47.793563 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.793543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8nsx\" (UniqueName: \"kubernetes.io/projected/0250feb6-d7bc-429e-bde9-d16d3e95ed96-kube-api-access-q8nsx\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:47.793742 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:47.793602    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 00:01:47.793742 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:47.793670    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0250feb6-d7bc-429e-bde9-d16d3e95ed96-cert podName:0250feb6-d7bc-429e-bde9-d16d3e95ed96 nodeName:}" failed. No retries permitted until 2026-07-07 00:01:48.293653391 +0000 UTC m=+340.706667281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0250feb6-d7bc-429e-bde9-d16d3e95ed96-cert") pod "odh-model-controller-7998c67dc4-f7xn7" (UID: "0250feb6-d7bc-429e-bde9-d16d3e95ed96") : secret "odh-model-controller-webhook-cert" not found
Jul 07 00:01:47.802882 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:47.802853    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8nsx\" (UniqueName: \"kubernetes.io/projected/0250feb6-d7bc-429e-bde9-d16d3e95ed96-kube-api-access-q8nsx\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:48.299135 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.299095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0250feb6-d7bc-429e-bde9-d16d3e95ed96-cert\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:48.301421 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.301401    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0250feb6-d7bc-429e-bde9-d16d3e95ed96-cert\") pod \"odh-model-controller-7998c67dc4-f7xn7\" (UID: \"0250feb6-d7bc-429e-bde9-d16d3e95ed96\") " pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:48.418690 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.418661    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:48.532878 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.532841    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-tsrs9"]
Jul 07 00:01:48.536839 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.536806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5" event={"ID":"cdb0be87-65eb-40b6-9c56-e07ddee42b88","Type":"ContainerStarted","Data":"8ff9cadfcf29ceccfc3dc575be1dd1ee8101d5fa012509cb1d09804d6f7dfd98"}
Jul 07 00:01:48.536994 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.536907    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.539761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.539735    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 00:01:48.539882 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.539782    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-txw94\""
Jul 07 00:01:48.547767 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.547663    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-tsrs9"]
Jul 07 00:01:48.565617 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.565591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-f7xn7"]
Jul 07 00:01:48.568681 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:48.568633    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0250feb6_d7bc_429e_bde9_d16d3e95ed96.slice/crio-e0f281d26cbd79f957344daf8d96d974d10c2e649e790ba6d7323d7ccf13218b WatchSource:0}: Error finding container e0f281d26cbd79f957344daf8d96d974d10c2e649e790ba6d7323d7ccf13218b: Status 404 returned error can't find the container with id e0f281d26cbd79f957344daf8d96d974d10c2e649e790ba6d7323d7ccf13218b
Jul 07 00:01:48.702624 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.702585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhcdf\" (UniqueName: \"kubernetes.io/projected/6c6914c8-2db9-46ad-b6d5-788ec2769ecc-kube-api-access-mhcdf\") pod \"model-serving-api-54d89bd5fc-tsrs9\" (UID: \"6c6914c8-2db9-46ad-b6d5-788ec2769ecc\") " pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.702826 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.702645    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6914c8-2db9-46ad-b6d5-788ec2769ecc-tls-certs\") pod \"model-serving-api-54d89bd5fc-tsrs9\" (UID: \"6c6914c8-2db9-46ad-b6d5-788ec2769ecc\") " pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.804011 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.803971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhcdf\" (UniqueName: \"kubernetes.io/projected/6c6914c8-2db9-46ad-b6d5-788ec2769ecc-kube-api-access-mhcdf\") pod \"model-serving-api-54d89bd5fc-tsrs9\" (UID: \"6c6914c8-2db9-46ad-b6d5-788ec2769ecc\") " pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.804011 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.804022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6914c8-2db9-46ad-b6d5-788ec2769ecc-tls-certs\") pod \"model-serving-api-54d89bd5fc-tsrs9\" (UID: \"6c6914c8-2db9-46ad-b6d5-788ec2769ecc\") " pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.806962 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.806934    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c6914c8-2db9-46ad-b6d5-788ec2769ecc-tls-certs\") pod \"model-serving-api-54d89bd5fc-tsrs9\" (UID: \"6c6914c8-2db9-46ad-b6d5-788ec2769ecc\") " pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.813045 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.813020    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhcdf\" (UniqueName: \"kubernetes.io/projected/6c6914c8-2db9-46ad-b6d5-788ec2769ecc-kube-api-access-mhcdf\") pod \"model-serving-api-54d89bd5fc-tsrs9\" (UID: \"6c6914c8-2db9-46ad-b6d5-788ec2769ecc\") " pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.853248 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.853213    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:48.980821 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.980788    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"]
Jul 07 00:01:48.984498 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.984469    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:48.987495 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.987228    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 00:01:48.987495 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.987371    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-78jhc\""
Jul 07 00:01:48.987938 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.987694    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 07 00:01:48.987938 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.987756    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 07 00:01:48.987938 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:48.987904    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 07 00:01:49.000943 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.000915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"]
Jul 07 00:01:49.004959 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.004934    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-tsrs9"]
Jul 07 00:01:49.008384 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:49.008350    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6914c8_2db9_46ad_b6d5_788ec2769ecc.slice/crio-c2feb3161173b74760646d109996a44bcf53c9b417bb5709ca6b97474544c8c8 WatchSource:0}: Error finding container c2feb3161173b74760646d109996a44bcf53c9b417bb5709ca6b97474544c8c8: Status 404 returned error can't find the container with id c2feb3161173b74760646d109996a44bcf53c9b417bb5709ca6b97474544c8c8
Jul 07 00:01:49.106702 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106615    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-token\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.106702 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.106937 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.106937 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp629\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-kube-api-access-lp629\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.106937 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.106937 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-local-certs\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.107145 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.106933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-cacerts\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.207860 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.207820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-token\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.208032 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.207880    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.208032 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.207940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.208032 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.207994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp629\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-kube-api-access-lp629\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.208189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.208046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.208189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.208096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-local-certs\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.208189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.208122    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-cacerts\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.209896 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.209288    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.211852 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.211824    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-cacerts\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.213018 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.212983    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.213281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.213243    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.214093 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.214051    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-local-certs\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.219043 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.219018    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp629\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-kube-api-access-lp629\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.219397 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.219374    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-token\") pod \"istiod-openshift-gateway-7574459656-5kz5w\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.299816 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.299701    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:49.479023 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.478989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"]
Jul 07 00:01:49.483122 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:49.483054    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e6a02a0_3ad0_4d6e_913a_0cabb0fdafed.slice/crio-bc926a6401350f50deec5dbca1f6abbe863ec53219c91ac74138f9dd099b62ad WatchSource:0}: Error finding container bc926a6401350f50deec5dbca1f6abbe863ec53219c91ac74138f9dd099b62ad: Status 404 returned error can't find the container with id bc926a6401350f50deec5dbca1f6abbe863ec53219c91ac74138f9dd099b62ad
Jul 07 00:01:49.542159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.542059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7" event={"ID":"0250feb6-d7bc-429e-bde9-d16d3e95ed96","Type":"ContainerStarted","Data":"e0f281d26cbd79f957344daf8d96d974d10c2e649e790ba6d7323d7ccf13218b"}
Jul 07 00:01:49.543965 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.543934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" event={"ID":"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed","Type":"ContainerStarted","Data":"bc926a6401350f50deec5dbca1f6abbe863ec53219c91ac74138f9dd099b62ad"}
Jul 07 00:01:49.546222 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:49.546193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9" event={"ID":"6c6914c8-2db9-46ad-b6d5-788ec2769ecc","Type":"ContainerStarted","Data":"c2feb3161173b74760646d109996a44bcf53c9b417bb5709ca6b97474544c8c8"}
Jul 07 00:01:52.563232 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:52.563176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7" event={"ID":"0250feb6-d7bc-429e-bde9-d16d3e95ed96","Type":"ContainerStarted","Data":"08bf3a80553efd822ddc02998e62a7e1776ea875c868e625fbe1df061338f0e5"}
Jul 07 00:01:52.563725 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:52.563254    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:01:52.565168 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:52.565130    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5" event={"ID":"cdb0be87-65eb-40b6-9c56-e07ddee42b88","Type":"ContainerStarted","Data":"5461ddb3665870dbdcf00e98e8d21a43dbaaf9fca47465df42c736eee9f4b811"}
Jul 07 00:01:52.580085 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:52.580023    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7" podStartSLOduration=2.526207129 podStartE2EDuration="5.580007315s" totalImagesPullingTime="3.053800186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:52.579166089 +0000 UTC m=+344.992179998" watchObservedRunningTime="2026-07-07 00:01:52.580007315 +0000 UTC m=+344.993021222"
Jul 07 00:01:52.596791 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:52.596735    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56d7555db7-x88l5" podStartSLOduration=1.780789103 podStartE2EDuration="5.596719071s" totalImagesPullingTime="3.815929968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:52.594763675 +0000 UTC m=+345.007777583" watchObservedRunningTime="2026-07-07 00:01:52.596719071 +0000 UTC m=+345.009732984"
Jul 07 00:01:53.356198 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.356151    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:01:53.356288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.356228    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:01:53.571346 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.571311    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" event={"ID":"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed","Type":"ContainerStarted","Data":"b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934"}
Jul 07 00:01:53.571847 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.571416    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:53.573004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.572975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9" event={"ID":"6c6914c8-2db9-46ad-b6d5-788ec2769ecc","Type":"ContainerStarted","Data":"5787bd4a73a8cb8792b1736bb63296983e816324e4c5aabd7c8c35786653649c"}
Jul 07 00:01:53.573136 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.573021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:01:53.593281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.593232    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" podStartSLOduration=1.723103589 podStartE2EDuration="5.59321806s" totalImagesPullingTime="3.870114471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:53.591465801 +0000 UTC m=+346.004479704" watchObservedRunningTime="2026-07-07 00:01:53.59321806 +0000 UTC m=+346.006231967"
Jul 07 00:01:53.610689 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:53.610594    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9" podStartSLOduration=1.266213651 podStartE2EDuration="5.61058086s" totalImagesPullingTime="4.344367209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:01:53.608981535 +0000 UTC m=+346.021995443" watchObservedRunningTime="2026-07-07 00:01:53.61058086 +0000 UTC m=+346.023594767"
Jul 07 00:01:54.581237 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:54.581200    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5kz5w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:01:54.581732 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:54.581262    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" podUID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:01:55.280240 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.280198    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"]
Jul 07 00:01:55.283818 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.283799    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.286506 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.286485    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zqbz7\""
Jul 07 00:01:55.295907 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.295885    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"]
Jul 07 00:01:55.355501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355663 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355663 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355663 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355663 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355663 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355846 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355846 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xws2x\" (UniqueName: \"kubernetes.io/projected/6609b6e5-bb86-4870-ae30-f1d6a11a661d-kube-api-access-xws2x\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.355846 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.355763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456352 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xws2x\" (UniqueName: \"kubernetes.io/projected/6609b6e5-bb86-4870-ae30-f1d6a11a661d-kube-api-access-xws2x\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456692 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456692 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456795 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456795 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456795 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456933 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456872    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.456981 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.456924    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.457177 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.457141    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.457269 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.457177    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.457640 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.457616    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.458998 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.458978    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.459142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.459125    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.465317 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.465279    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6609b6e5-bb86-4870-ae30-f1d6a11a661d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.465558 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.465530    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xws2x\" (UniqueName: \"kubernetes.io/projected/6609b6e5-bb86-4870-ae30-f1d6a11a661d-kube-api-access-xws2x\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678w6qh2\" (UID: \"6609b6e5-bb86-4870-ae30-f1d6a11a661d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.593770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.593747    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:01:55.732274 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:55.732228    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"]
Jul 07 00:01:55.735534 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:55.735504    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6609b6e5_bb86_4870_ae30_f1d6a11a661d.slice/crio-0f64061026639ea991abf0ef1f509719fd07f06d6b132ce2419681d6ca548a15 WatchSource:0}: Error finding container 0f64061026639ea991abf0ef1f509719fd07f06d6b132ce2419681d6ca548a15: Status 404 returned error can't find the container with id 0f64061026639ea991abf0ef1f509719fd07f06d6b132ce2419681d6ca548a15
Jul 07 00:01:56.590807 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:56.590775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2" event={"ID":"6609b6e5-bb86-4870-ae30-f1d6a11a661d","Type":"ContainerStarted","Data":"0f64061026639ea991abf0ef1f509719fd07f06d6b132ce2419681d6ca548a15"}
Jul 07 00:01:57.580743 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.580715    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:01:57.840038 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.840008    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"]
Jul 07 00:01:57.844042 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.844024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:57.851497 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.851448    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 07 00:01:57.851604 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.851525    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-g489g\""
Jul 07 00:01:57.861103 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.861081    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"]
Jul 07 00:01:57.870028 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.870000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"]
Jul 07 00:01:57.873368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.873347    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:57.876206 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.876186    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p5ss7\""
Jul 07 00:01:57.876368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.876352    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 07 00:01:57.880373 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.880348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlrmg\" (UniqueName: \"kubernetes.io/projected/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-kube-api-access-jlrmg\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:57.880506 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.880411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:57.887019 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.886994    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"]
Jul 07 00:01:57.981141 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.981102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:57.981322 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.981153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3cd4eaf-58f0-4620-931f-6631a3d2d229-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:57.981322 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.981200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrg5\" (UniqueName: \"kubernetes.io/projected/b3cd4eaf-58f0-4620-931f-6631a3d2d229-kube-api-access-cvrg5\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:57.981322 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.981231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlrmg\" (UniqueName: \"kubernetes.io/projected/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-kube-api-access-jlrmg\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:57.981322 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:57.981249    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 07 00:01:57.981484 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:57.981324    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert podName:a7d5d300-2afb-4e36-b6b7-62846e3aa0fc nodeName:}" failed. No retries permitted until 2026-07-07 00:01:58.481300288 +0000 UTC m=+350.894314261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert") pod "kserve-controller-manager-d6f4b65b8-rb8zq" (UID: "a7d5d300-2afb-4e36-b6b7-62846e3aa0fc") : secret "kserve-webhook-server-cert" not found
Jul 07 00:01:57.993082 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:57.993047    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlrmg\" (UniqueName: \"kubernetes.io/projected/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-kube-api-access-jlrmg\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:58.082094 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.082050    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3cd4eaf-58f0-4620-931f-6631a3d2d229-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:58.082296 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.082113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrg5\" (UniqueName: \"kubernetes.io/projected/b3cd4eaf-58f0-4620-931f-6631a3d2d229-kube-api-access-cvrg5\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:58.082296 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:58.082219    2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 07 00:01:58.082418 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:58.082299    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3cd4eaf-58f0-4620-931f-6631a3d2d229-cert podName:b3cd4eaf-58f0-4620-931f-6631a3d2d229 nodeName:}" failed. No retries permitted until 2026-07-07 00:01:58.582279944 +0000 UTC m=+350.995293843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b3cd4eaf-58f0-4620-931f-6631a3d2d229-cert") pod "llmisvc-controller-manager-777c6bd7d5-fpsgs" (UID: "b3cd4eaf-58f0-4620-931f-6631a3d2d229") : secret "llmisvc-webhook-server-cert" not found
Jul 07 00:01:58.094058 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.093998    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrg5\" (UniqueName: \"kubernetes.io/projected/b3cd4eaf-58f0-4620-931f-6631a3d2d229-kube-api-access-cvrg5\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:58.485325 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.485238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:58.485499 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:58.485364    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 07 00:01:58.485499 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:01:58.485417    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert podName:a7d5d300-2afb-4e36-b6b7-62846e3aa0fc nodeName:}" failed. No retries permitted until 2026-07-07 00:01:59.485404225 +0000 UTC m=+351.898418111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert") pod "kserve-controller-manager-d6f4b65b8-rb8zq" (UID: "a7d5d300-2afb-4e36-b6b7-62846e3aa0fc") : secret "kserve-webhook-server-cert" not found
Jul 07 00:01:58.585636 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.585601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3cd4eaf-58f0-4620-931f-6631a3d2d229-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:58.588044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.588018    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3cd4eaf-58f0-4620-931f-6631a3d2d229-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fpsgs\" (UID: \"b3cd4eaf-58f0-4620-931f-6631a3d2d229\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:58.783379 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.783306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:01:58.913228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:58.913200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"]
Jul 07 00:01:58.915792 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:01:58.915747    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb3cd4eaf_58f0_4620_931f_6631a3d2d229.slice/crio-09c528820446041664fbd2aa2c6037ba2414cccfacde13428f0aec73dd74619f WatchSource:0}: Error finding container 09c528820446041664fbd2aa2c6037ba2414cccfacde13428f0aec73dd74619f: Status 404 returned error can't find the container with id 09c528820446041664fbd2aa2c6037ba2414cccfacde13428f0aec73dd74619f
Jul 07 00:01:59.492959 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:59.492925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:59.495492 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:59.495415    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7d5d300-2afb-4e36-b6b7-62846e3aa0fc-cert\") pod \"kserve-controller-manager-d6f4b65b8-rb8zq\" (UID: \"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:01:59.605656 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:59.605619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs" event={"ID":"b3cd4eaf-58f0-4620-931f-6631a3d2d229","Type":"ContainerStarted","Data":"09c528820446041664fbd2aa2c6037ba2414cccfacde13428f0aec73dd74619f"}
Jul 07 00:01:59.654868 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:01:59.654835    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:02:00.890761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:00.890737    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"]
Jul 07 00:02:00.912080 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:02:00.912051    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d5d300_2afb_4e36_b6b7_62846e3aa0fc.slice/crio-6e0517ed2c6f47dcbf31e0ac187c31109ee76f9d8dfbb6f4444a22d18764d0ab WatchSource:0}: Error finding container 6e0517ed2c6f47dcbf31e0ac187c31109ee76f9d8dfbb6f4444a22d18764d0ab: Status 404 returned error can't find the container with id 6e0517ed2c6f47dcbf31e0ac187c31109ee76f9d8dfbb6f4444a22d18764d0ab
Jul 07 00:02:00.990754 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:00.990696    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:02:00.990908 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:00.990791    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:02:00.990908 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:00.990834    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:02:01.187550 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.187511    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"]
Jul 07 00:02:01.187743 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.187720    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" podUID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" containerName="discovery" containerID="cri-o://b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934" gracePeriod=30
Jul 07 00:02:01.434950 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.434922    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:02:01.511895 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.511855    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp629\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-kube-api-access-lp629\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.512076 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.511917    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-dns-cert\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.512076 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.511948    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-kubeconfig\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.512076 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.512019    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-local-certs\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.512076 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.512062    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-ca-configmap\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.512381 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.512101    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-token\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.512381 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.512144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-cacerts\") pod \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\" (UID: \"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed\") "
Jul 07 00:02:01.514319 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.514288    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:02:01.516875 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.516845    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:02:01.516986 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.516954    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-cacerts" (OuterVolumeSpecName: "cacerts") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:02:01.517110 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.517059    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-kube-api-access-lp629" (OuterVolumeSpecName: "kube-api-access-lp629") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "kube-api-access-lp629". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:02:01.517230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.517159    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-local-certs" (OuterVolumeSpecName: "local-certs") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:02:01.517553 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.517520    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:02:01.518440 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.518402    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" (UID: "0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.612915    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-local-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.612954    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-ca-configmap\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.612970    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.612983    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-cacerts\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.612996    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp629\" (UniqueName: \"kubernetes.io/projected/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-kube-api-access-lp629\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.613008    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-csr-dns-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.613281 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.613021    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed-istio-kubeconfig\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:01.618069 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.617962    2564 generic.go:356] "Generic (PLEG): container finished" podID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" containerID="b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934" exitCode=0
Jul 07 00:02:01.618194 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.618080    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"
Jul 07 00:02:01.618194 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.618084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" event={"ID":"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed","Type":"ContainerDied","Data":"b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934"}
Jul 07 00:02:01.618194 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.618159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w" event={"ID":"0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed","Type":"ContainerDied","Data":"bc926a6401350f50deec5dbca1f6abbe863ec53219c91ac74138f9dd099b62ad"}
Jul 07 00:02:01.618194 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.618190    2564 scope.go:117] "RemoveContainer" containerID="b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934"
Jul 07 00:02:01.622323 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.621184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2" event={"ID":"6609b6e5-bb86-4870-ae30-f1d6a11a661d","Type":"ContainerStarted","Data":"b8f75e76ce8b48a7cdaac53759d1ba2fda76322290422518dc58048465b5d1a3"}
Jul 07 00:02:01.623362 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.623338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq" event={"ID":"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc","Type":"ContainerStarted","Data":"6e0517ed2c6f47dcbf31e0ac187c31109ee76f9d8dfbb6f4444a22d18764d0ab"}
Jul 07 00:02:01.631820 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.631802    2564 scope.go:117] "RemoveContainer" containerID="b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934"
Jul 07 00:02:01.632163 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:02:01.632138    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934\": container with ID starting with b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934 not found: ID does not exist" containerID="b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934"
Jul 07 00:02:01.632230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.632174    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934"} err="failed to get container status \"b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934\": rpc error: code = NotFound desc = could not find container \"b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934\": container with ID starting with b7c5d0ec9d0b7c2eea5b4f5b56102532bfd24232b9cc96fbeadd6a03dc962934 not found: ID does not exist"
Jul 07 00:02:01.644220 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.644173    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2" podStartSLOduration=1.39136729 podStartE2EDuration="6.644154986s" totalImagesPullingTime="5.252787696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:01.641719968 +0000 UTC m=+354.054733887" watchObservedRunningTime="2026-07-07 00:02:01.644154986 +0000 UTC m=+354.057168899"
Jul 07 00:02:01.656192 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.656167    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"]
Jul 07 00:02:01.663497 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:01.663474    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5kz5w"]
Jul 07 00:02:02.116199 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:02.116167    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" path="/var/lib/kubelet/pods/0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed/volumes"
Jul 07 00:02:02.594532 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:02.594497    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:02:02.599208 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:02.599185    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:02:02.628867 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:02.628841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:02:02.629877 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:02.629850    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678w6qh2"
Jul 07 00:02:03.575843 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.575759    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-f7xn7"
Jul 07 00:02:03.639307 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.639275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq" event={"ID":"a7d5d300-2afb-4e36-b6b7-62846e3aa0fc","Type":"ContainerStarted","Data":"385d25b133f76536f12df5ab2f2a8905ca860bde83ae27e8d8317b4bc6d72396"}
Jul 07 00:02:03.639523 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.639351    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:02:03.640816 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.640789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs" event={"ID":"b3cd4eaf-58f0-4620-931f-6631a3d2d229","Type":"ContainerStarted","Data":"9ba67ecf513f6cbe88b01aa92e73b8f0e1ca0cc8eefc79f7fa1771a4f5c4283a"}
Jul 07 00:02:03.640998 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.640986    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:02:03.678432 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.678386    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq" podStartSLOduration=4.395334616 podStartE2EDuration="6.678370853s" totalImagesPullingTime="2.283036237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:03.676881982 +0000 UTC m=+356.089895891" watchObservedRunningTime="2026-07-07 00:02:03.678370853 +0000 UTC m=+356.091384760"
Jul 07 00:02:03.734402 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:03.734350    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs" podStartSLOduration=2.457467747 podStartE2EDuration="6.734334373s" totalImagesPullingTime="4.276866626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:03.733912583 +0000 UTC m=+356.146926504" watchObservedRunningTime="2026-07-07 00:02:03.734334373 +0000 UTC m=+356.147348281"
Jul 07 00:02:04.583277 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:04.583245    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-tsrs9"
Jul 07 00:02:34.647145 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:34.647115    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fpsgs"
Jul 07 00:02:34.650188 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:34.650167    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rb8zq"
Jul 07 00:02:48.349047 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.349005    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2c97r"]
Jul 07 00:02:48.349508 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.349362    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" containerName="discovery"
Jul 07 00:02:48.349508 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.349376    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" containerName="discovery"
Jul 07 00:02:48.349508 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.349479    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e6a02a0-3ad0-4d6e-913a-0cabb0fdafed" containerName="discovery"
Jul 07 00:02:48.357792 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.357767    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:48.362111 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.362090    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 00:02:48.363278 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.363253    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-54k5t\""
Jul 07 00:02:48.363398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.363259    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 00:02:48.375321 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.375299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2c97r"]
Jul 07 00:02:48.488557 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.488519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrgp\" (UniqueName: \"kubernetes.io/projected/f22bf778-308e-494f-9a7b-e0fd4e93931b-kube-api-access-jqrgp\") pod \"kuadrant-operator-catalog-2c97r\" (UID: \"f22bf778-308e-494f-9a7b-e0fd4e93931b\") " pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:48.589364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.589324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrgp\" (UniqueName: \"kubernetes.io/projected/f22bf778-308e-494f-9a7b-e0fd4e93931b-kube-api-access-jqrgp\") pod \"kuadrant-operator-catalog-2c97r\" (UID: \"f22bf778-308e-494f-9a7b-e0fd4e93931b\") " pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:48.599762 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.599701    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrgp\" (UniqueName: \"kubernetes.io/projected/f22bf778-308e-494f-9a7b-e0fd4e93931b-kube-api-access-jqrgp\") pod \"kuadrant-operator-catalog-2c97r\" (UID: \"f22bf778-308e-494f-9a7b-e0fd4e93931b\") " pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:48.669156 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.669126    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:48.689396 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.689370    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2c97r"]
Jul 07 00:02:48.794279 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.794228    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2c97r"]
Jul 07 00:02:48.797171 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:02:48.797142    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22bf778_308e_494f_9a7b_e0fd4e93931b.slice/crio-7389ffef7c5d7ca40a24961b48953e0f7e327ff5dc3b78a8c5a88a4cbc54f4bf WatchSource:0}: Error finding container 7389ffef7c5d7ca40a24961b48953e0f7e327ff5dc3b78a8c5a88a4cbc54f4bf: Status 404 returned error can't find the container with id 7389ffef7c5d7ca40a24961b48953e0f7e327ff5dc3b78a8c5a88a4cbc54f4bf
Jul 07 00:02:48.836634 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.836566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2c97r" event={"ID":"f22bf778-308e-494f-9a7b-e0fd4e93931b","Type":"ContainerStarted","Data":"7389ffef7c5d7ca40a24961b48953e0f7e327ff5dc3b78a8c5a88a4cbc54f4bf"}
Jul 07 00:02:48.897014 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.896955    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p65st"]
Jul 07 00:02:48.901255 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.901241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:48.909733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:48.909712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p65st"]
Jul 07 00:02:49.093424 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:49.093393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96x9n\" (UniqueName: \"kubernetes.io/projected/c3c0e3f7-d421-4c6c-ba89-117db18e2778-kube-api-access-96x9n\") pod \"kuadrant-operator-catalog-p65st\" (UID: \"c3c0e3f7-d421-4c6c-ba89-117db18e2778\") " pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:49.194687 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:49.194611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96x9n\" (UniqueName: \"kubernetes.io/projected/c3c0e3f7-d421-4c6c-ba89-117db18e2778-kube-api-access-96x9n\") pod \"kuadrant-operator-catalog-p65st\" (UID: \"c3c0e3f7-d421-4c6c-ba89-117db18e2778\") " pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:49.203159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:49.203134    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96x9n\" (UniqueName: \"kubernetes.io/projected/c3c0e3f7-d421-4c6c-ba89-117db18e2778-kube-api-access-96x9n\") pod \"kuadrant-operator-catalog-p65st\" (UID: \"c3c0e3f7-d421-4c6c-ba89-117db18e2778\") " pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:49.211081 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:49.210952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:49.355680 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:49.355657    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p65st"]
Jul 07 00:02:49.403664 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:02:49.403632    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c0e3f7_d421_4c6c_ba89_117db18e2778.slice/crio-5122764f11d0ae6a007fd3dcc4d62d1b900df4ddf75623b025ea0f8cb69e62e8 WatchSource:0}: Error finding container 5122764f11d0ae6a007fd3dcc4d62d1b900df4ddf75623b025ea0f8cb69e62e8: Status 404 returned error can't find the container with id 5122764f11d0ae6a007fd3dcc4d62d1b900df4ddf75623b025ea0f8cb69e62e8
Jul 07 00:02:49.843892 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:49.843859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p65st" event={"ID":"c3c0e3f7-d421-4c6c-ba89-117db18e2778","Type":"ContainerStarted","Data":"5122764f11d0ae6a007fd3dcc4d62d1b900df4ddf75623b025ea0f8cb69e62e8"}
Jul 07 00:02:50.850986 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:50.850912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2c97r" event={"ID":"f22bf778-308e-494f-9a7b-e0fd4e93931b","Type":"ContainerStarted","Data":"a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447"}
Jul 07 00:02:50.850986 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:50.850956    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2c97r" podUID="f22bf778-308e-494f-9a7b-e0fd4e93931b" containerName="registry-server" containerID="cri-o://a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447" gracePeriod=2
Jul 07 00:02:50.852715 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:50.852687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p65st" event={"ID":"c3c0e3f7-d421-4c6c-ba89-117db18e2778","Type":"ContainerStarted","Data":"3ed104ab12b69ff0d1e92bdc3489f9c53ff89d17616b41e13f10bb7f4b4f2e25"}
Jul 07 00:02:50.868491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:50.868417    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2c97r" podStartSLOduration=0.955681753 podStartE2EDuration="2.868399294s" totalImagesPullingTime="1.912717541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:50.867306619 +0000 UTC m=+403.280320540" watchObservedRunningTime="2026-07-07 00:02:50.868399294 +0000 UTC m=+403.281413204"
Jul 07 00:02:50.884221 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:50.884175    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-p65st" podStartSLOduration=1.5773910469999999 podStartE2EDuration="2.884160417s" totalImagesPullingTime="1.30676937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:50.883431839 +0000 UTC m=+403.296445777" watchObservedRunningTime="2026-07-07 00:02:50.884160417 +0000 UTC m=+403.297174325"
Jul 07 00:02:51.190847 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.190822    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:51.211794 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.211772    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqrgp\" (UniqueName: \"kubernetes.io/projected/f22bf778-308e-494f-9a7b-e0fd4e93931b-kube-api-access-jqrgp\") pod \"f22bf778-308e-494f-9a7b-e0fd4e93931b\" (UID: \"f22bf778-308e-494f-9a7b-e0fd4e93931b\") "
Jul 07 00:02:51.213838 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.213814    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22bf778-308e-494f-9a7b-e0fd4e93931b-kube-api-access-jqrgp" (OuterVolumeSpecName: "kube-api-access-jqrgp") pod "f22bf778-308e-494f-9a7b-e0fd4e93931b" (UID: "f22bf778-308e-494f-9a7b-e0fd4e93931b"). InnerVolumeSpecName "kube-api-access-jqrgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:02:51.312681 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.312608    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqrgp\" (UniqueName: \"kubernetes.io/projected/f22bf778-308e-494f-9a7b-e0fd4e93931b-kube-api-access-jqrgp\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:02:51.858560 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.858527    2564 generic.go:356] "Generic (PLEG): container finished" podID="f22bf778-308e-494f-9a7b-e0fd4e93931b" containerID="a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447" exitCode=0
Jul 07 00:02:51.858997 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.858567    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2c97r"
Jul 07 00:02:51.858997 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.858611    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2c97r" event={"ID":"f22bf778-308e-494f-9a7b-e0fd4e93931b","Type":"ContainerDied","Data":"a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447"}
Jul 07 00:02:51.858997 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.858664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2c97r" event={"ID":"f22bf778-308e-494f-9a7b-e0fd4e93931b","Type":"ContainerDied","Data":"7389ffef7c5d7ca40a24961b48953e0f7e327ff5dc3b78a8c5a88a4cbc54f4bf"}
Jul 07 00:02:51.858997 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.858685    2564 scope.go:117] "RemoveContainer" containerID="a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447"
Jul 07 00:02:51.868107 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.868091    2564 scope.go:117] "RemoveContainer" containerID="a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447"
Jul 07 00:02:51.868334 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:02:51.868317    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447\": container with ID starting with a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447 not found: ID does not exist" containerID="a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447"
Jul 07 00:02:51.868386 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.868342    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447"} err="failed to get container status \"a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447\": rpc error: code = NotFound desc = could not find container \"a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447\": container with ID starting with a05e2066dc4c7301c0c26628154e0d98877b756fa42311e052e85e6066cf4447 not found: ID does not exist"
Jul 07 00:02:51.880492 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.880470    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2c97r"]
Jul 07 00:02:51.885276 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:51.885257    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2c97r"]
Jul 07 00:02:52.114564 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:52.114529    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f22bf778-308e-494f-9a7b-e0fd4e93931b" path="/var/lib/kubelet/pods/f22bf778-308e-494f-9a7b-e0fd4e93931b/volumes"
Jul 07 00:02:59.211334 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:59.211288    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:59.211936 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:59.211361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:59.233112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:59.233089    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:02:59.913142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:02:59.913118    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-p65st"
Jul 07 00:03:14.531659 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.531615    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"]
Jul 07 00:03:14.532044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.531929    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f22bf778-308e-494f-9a7b-e0fd4e93931b" containerName="registry-server"
Jul 07 00:03:14.532044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.531941    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f22bf778-308e-494f-9a7b-e0fd4e93931b" containerName="registry-server"
Jul 07 00:03:14.532044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.532008    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f22bf778-308e-494f-9a7b-e0fd4e93931b" containerName="registry-server"
Jul 07 00:03:14.539205 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.539183    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:14.542676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.542655    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-95g4r\""
Jul 07 00:03:14.551695 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.551671    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"]
Jul 07 00:03:14.598505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.598481    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2hvh\" (UniqueName: \"kubernetes.io/projected/e35a51df-9478-4312-a105-c0802eed8b32-kube-api-access-d2hvh\") pod \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" (UID: \"e35a51df-9478-4312-a105-c0802eed8b32\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:14.699521 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.699481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2hvh\" (UniqueName: \"kubernetes.io/projected/e35a51df-9478-4312-a105-c0802eed8b32-kube-api-access-d2hvh\") pod \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" (UID: \"e35a51df-9478-4312-a105-c0802eed8b32\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:14.719987 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.719964    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2hvh\" (UniqueName: \"kubernetes.io/projected/e35a51df-9478-4312-a105-c0802eed8b32-kube-api-access-d2hvh\") pod \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" (UID: \"e35a51df-9478-4312-a105-c0802eed8b32\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:14.848770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:14.848737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:15.012921 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:15.012891    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"]
Jul 07 00:03:15.014861 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:03:15.014835    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35a51df_9478_4312_a105_c0802eed8b32.slice/crio-4ae5383633eed082429c7306119eacfd701a15cb625b510c024897e7b6c796ea WatchSource:0}: Error finding container 4ae5383633eed082429c7306119eacfd701a15cb625b510c024897e7b6c796ea: Status 404 returned error can't find the container with id 4ae5383633eed082429c7306119eacfd701a15cb625b510c024897e7b6c796ea
Jul 07 00:03:15.959084 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:15.959047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l" event={"ID":"e35a51df-9478-4312-a105-c0802eed8b32","Type":"ContainerStarted","Data":"4ae5383633eed082429c7306119eacfd701a15cb625b510c024897e7b6c796ea"}
Jul 07 00:03:17.971533 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:17.971496    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l" event={"ID":"e35a51df-9478-4312-a105-c0802eed8b32","Type":"ContainerStarted","Data":"93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1"}
Jul 07 00:03:17.971909 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:17.971578    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:17.996927 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:17.996882    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l" podStartSLOduration=2.100615536 podStartE2EDuration="3.996867029s" totalImagesPullingTime="1.896251493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:17.996212725 +0000 UTC m=+430.409226674" watchObservedRunningTime="2026-07-07 00:03:17.996867029 +0000 UTC m=+430.409880939"
Jul 07 00:03:20.013273 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.013241    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"]
Jul 07 00:03:20.016606 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.016589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.020650 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.020633    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9glnk\""
Jul 07 00:03:20.037816 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.037794    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"]
Jul 07 00:03:20.146748 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.146720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szc8\" (UniqueName: \"kubernetes.io/projected/bcb0a02b-a4bf-4835-b067-6f88fd7db071-kube-api-access-4szc8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.146879 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.146778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcb0a02b-a4bf-4835-b067-6f88fd7db071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.247303 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.247273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szc8\" (UniqueName: \"kubernetes.io/projected/bcb0a02b-a4bf-4835-b067-6f88fd7db071-kube-api-access-4szc8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.247532 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.247363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcb0a02b-a4bf-4835-b067-6f88fd7db071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.247863 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.247841    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcb0a02b-a4bf-4835-b067-6f88fd7db071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.259581 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.259554    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szc8\" (UniqueName: \"kubernetes.io/projected/bcb0a02b-a4bf-4835-b067-6f88fd7db071-kube-api-access-4szc8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.326223 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.326163    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:20.471163 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.471143    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"]
Jul 07 00:03:20.473654 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:03:20.473626    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb0a02b_a4bf_4835_b067_6f88fd7db071.slice/crio-63f7378b12d0a6806680eb5fc1f4238cf7ca3dd0f13fa64eff165eb57ec2d217 WatchSource:0}: Error finding container 63f7378b12d0a6806680eb5fc1f4238cf7ca3dd0f13fa64eff165eb57ec2d217: Status 404 returned error can't find the container with id 63f7378b12d0a6806680eb5fc1f4238cf7ca3dd0f13fa64eff165eb57ec2d217
Jul 07 00:03:20.985966 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:20.985925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5" event={"ID":"bcb0a02b-a4bf-4835-b067-6f88fd7db071","Type":"ContainerStarted","Data":"63f7378b12d0a6806680eb5fc1f4238cf7ca3dd0f13fa64eff165eb57ec2d217"}
Jul 07 00:03:23.493171 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.493133    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"]
Jul 07 00:03:23.497517 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.497493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:23.501163 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.501138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s6kr9\""
Jul 07 00:03:23.522687 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.522659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"]
Jul 07 00:03:23.578720 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.578694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnlmk\" (UniqueName: \"kubernetes.io/projected/e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4-kube-api-access-wnlmk\") pod \"authorino-operator-75cc8d5cb7-nxjhp\" (UID: \"e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:23.679831 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.679794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnlmk\" (UniqueName: \"kubernetes.io/projected/e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4-kube-api-access-wnlmk\") pod \"authorino-operator-75cc8d5cb7-nxjhp\" (UID: \"e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:23.695142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.695112    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnlmk\" (UniqueName: \"kubernetes.io/projected/e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4-kube-api-access-wnlmk\") pod \"authorino-operator-75cc8d5cb7-nxjhp\" (UID: \"e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:23.809745 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:23.809667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:25.074491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:25.073078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"]
Jul 07 00:03:25.080325 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:03:25.080278    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b3a7ef_6c9c_469d_a91d_65fa2a049fb4.slice/crio-d60a37a82c659e8a55eb703d9483893b9dc02630cd37ab1e2153e2a6f237026a WatchSource:0}: Error finding container d60a37a82c659e8a55eb703d9483893b9dc02630cd37ab1e2153e2a6f237026a: Status 404 returned error can't find the container with id d60a37a82c659e8a55eb703d9483893b9dc02630cd37ab1e2153e2a6f237026a
Jul 07 00:03:26.011752 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:26.011715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp" event={"ID":"e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4","Type":"ContainerStarted","Data":"d60a37a82c659e8a55eb703d9483893b9dc02630cd37ab1e2153e2a6f237026a"}
Jul 07 00:03:26.013829 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:26.013796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5" event={"ID":"bcb0a02b-a4bf-4835-b067-6f88fd7db071","Type":"ContainerStarted","Data":"5019eff6457500a4c3ccde31b27cc79cf7c1ef267fdd3ad042a18635d440a146"}
Jul 07 00:03:26.014280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:26.014256    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:26.074682 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:26.074582    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5" podStartSLOduration=2.564603412 podStartE2EDuration="7.074561545s" totalImagesPullingTime="4.509958133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:26.068838685 +0000 UTC m=+438.481852593" watchObservedRunningTime="2026-07-07 00:03:26.074561545 +0000 UTC m=+438.487575456"
Jul 07 00:03:28.024949 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:28.024900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp" event={"ID":"e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4","Type":"ContainerStarted","Data":"6504963209d08d4ec7972df8bd3abda0e44956b26d49fd1ae98f15e9e29df63d"}
Jul 07 00:03:28.025335 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:28.025100    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:28.048809 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:28.048762    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp" podStartSLOduration=2.959570414 podStartE2EDuration="5.048748738s" totalImagesPullingTime="2.089178324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:28.046641548 +0000 UTC m=+440.459655474" watchObservedRunningTime="2026-07-07 00:03:28.048748738 +0000 UTC m=+440.461762645"
Jul 07 00:03:28.978765 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:28.978739    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:37.021062 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:37.020980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:38.958414 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:38.958379    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"]
Jul 07 00:03:38.958881 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:38.958567    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" containerName="manager" containerID="cri-o://5019eff6457500a4c3ccde31b27cc79cf7c1ef267fdd3ad042a18635d440a146" gracePeriod=2
Jul 07 00:03:38.968013 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:38.967985    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"]
Jul 07 00:03:38.991019 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:38.990995    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"]
Jul 07 00:03:38.991217 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:38.991192    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l" podUID="e35a51df-9478-4312-a105-c0802eed8b32" containerName="manager" containerID="cri-o://93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1" gracePeriod=2
Jul 07 00:03:38.994430 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:38.994402    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:39.002859 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.002832    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"]
Jul 07 00:03:39.016837 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.016811    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"]
Jul 07 00:03:39.017131 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.017119    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" containerName="manager"
Jul 07 00:03:39.017131 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.017132    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" containerName="manager"
Jul 07 00:03:39.017259 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.017143    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e35a51df-9478-4312-a105-c0802eed8b32" containerName="manager"
Jul 07 00:03:39.017259 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.017149    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e35a51df-9478-4312-a105-c0802eed8b32" containerName="manager"
Jul 07 00:03:39.017259 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.017211    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" containerName="manager"
Jul 07 00:03:39.017259 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.017220    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e35a51df-9478-4312-a105-c0802eed8b32" containerName="manager"
Jul 07 00:03:39.020123 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.020105    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.024031 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.024002    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="e35a51df-9478-4312-a105-c0802eed8b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:39.035136 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.035117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nxjhp"
Jul 07 00:03:39.039512 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.039493    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"]
Jul 07 00:03:39.042617 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.042602    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:39.068733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.068709    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"]
Jul 07 00:03:39.079385 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.079363    2564 generic.go:356] "Generic (PLEG): container finished" podID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" containerID="5019eff6457500a4c3ccde31b27cc79cf7c1ef267fdd3ad042a18635d440a146" exitCode=0
Jul 07 00:03:39.080511 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.080484    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"]
Jul 07 00:03:39.107466 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.107422    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:39.121143 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.121113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqf4v\" (UniqueName: \"kubernetes.io/projected/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-kube-api-access-qqf4v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h8fpp\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.121292 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.121187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h8fpp\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.221743 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.221651    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:39.221903 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.221670    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqf4v\" (UniqueName: \"kubernetes.io/projected/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-kube-api-access-qqf4v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h8fpp\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.222005 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.221986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h8fpp\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.222186 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.222112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjv2\" (UniqueName: \"kubernetes.io/projected/e64af115-de23-4f39-9bc6-eb09bdb125ca-kube-api-access-jbjv2\") pod \"limitador-operator-controller-manager-859d7b78bd-lfx9p\" (UID: \"e64af115-de23-4f39-9bc6-eb09bdb125ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:39.222393 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.222366    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h8fpp\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.225287 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.225261    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="e35a51df-9478-4312-a105-c0802eed8b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:39.251465 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.251436    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:39.253988 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.253973    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:39.254293 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.254275    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:39.257698 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.257664    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="e35a51df-9478-4312-a105-c0802eed8b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:39.260196 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.260173    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="e35a51df-9478-4312-a105-c0802eed8b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:39.270902 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:39.270877    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:39.273071 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.273053    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqf4v\" (UniqueName: \"kubernetes.io/projected/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-kube-api-access-qqf4v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h8fpp\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.323384 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.323360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjv2\" (UniqueName: \"kubernetes.io/projected/e64af115-de23-4f39-9bc6-eb09bdb125ca-kube-api-access-jbjv2\") pod \"limitador-operator-controller-manager-859d7b78bd-lfx9p\" (UID: \"e64af115-de23-4f39-9bc6-eb09bdb125ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:39.339573 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.339552    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjv2\" (UniqueName: \"kubernetes.io/projected/e64af115-de23-4f39-9bc6-eb09bdb125ca-kube-api-access-jbjv2\") pod \"limitador-operator-controller-manager-859d7b78bd-lfx9p\" (UID: \"e64af115-de23-4f39-9bc6-eb09bdb125ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:39.389534 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.389505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:39.395463 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.395431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:39.424571 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.424540    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2hvh\" (UniqueName: \"kubernetes.io/projected/e35a51df-9478-4312-a105-c0802eed8b32-kube-api-access-d2hvh\") pod \"e35a51df-9478-4312-a105-c0802eed8b32\" (UID: \"e35a51df-9478-4312-a105-c0802eed8b32\") "
Jul 07 00:03:39.424701 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.424633    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4szc8\" (UniqueName: \"kubernetes.io/projected/bcb0a02b-a4bf-4835-b067-6f88fd7db071-kube-api-access-4szc8\") pod \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") "
Jul 07 00:03:39.424701 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.424683    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcb0a02b-a4bf-4835-b067-6f88fd7db071-extensions-socket-volume\") pod \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\" (UID: \"bcb0a02b-a4bf-4835-b067-6f88fd7db071\") "
Jul 07 00:03:39.425283 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.425251    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb0a02b-a4bf-4835-b067-6f88fd7db071-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bcb0a02b-a4bf-4835-b067-6f88fd7db071" (UID: "bcb0a02b-a4bf-4835-b067-6f88fd7db071"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:03:39.426704 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.426677    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb0a02b-a4bf-4835-b067-6f88fd7db071-kube-api-access-4szc8" (OuterVolumeSpecName: "kube-api-access-4szc8") pod "bcb0a02b-a4bf-4835-b067-6f88fd7db071" (UID: "bcb0a02b-a4bf-4835-b067-6f88fd7db071"). InnerVolumeSpecName "kube-api-access-4szc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:03:39.426806 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.426790    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e35a51df-9478-4312-a105-c0802eed8b32-kube-api-access-d2hvh" (OuterVolumeSpecName: "kube-api-access-d2hvh") pod "e35a51df-9478-4312-a105-c0802eed8b32" (UID: "e35a51df-9478-4312-a105-c0802eed8b32"). InnerVolumeSpecName "kube-api-access-d2hvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:03:39.526018 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.525986    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcb0a02b-a4bf-4835-b067-6f88fd7db071-extensions-socket-volume\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:03:39.526018 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.526020    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2hvh\" (UniqueName: \"kubernetes.io/projected/e35a51df-9478-4312-a105-c0802eed8b32-kube-api-access-d2hvh\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:03:39.526222 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.526036    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4szc8\" (UniqueName: \"kubernetes.io/projected/bcb0a02b-a4bf-4835-b067-6f88fd7db071-kube-api-access-4szc8\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:03:39.555748 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.555726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"]
Jul 07 00:03:39.557584 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:03:39.557559    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aacd94b_8b9d_4648_b88d_60a4d59c5f66.slice/crio-0eae5a8abf2a193a0d4053ded0a4da14827d0c7ebeb8e62c346902e2c7039a87 WatchSource:0}: Error finding container 0eae5a8abf2a193a0d4053ded0a4da14827d0c7ebeb8e62c346902e2c7039a87: Status 404 returned error can't find the container with id 0eae5a8abf2a193a0d4053ded0a4da14827d0c7ebeb8e62c346902e2c7039a87
Jul 07 00:03:39.564309 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:39.564285    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"]
Jul 07 00:03:39.566930 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:03:39.566905    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode64af115_de23_4f39_9bc6_eb09bdb125ca.slice/crio-b283ec2bf84bb2c561641cfb5b7d5076833d9be72b69a716cc291680238afcb0 WatchSource:0}: Error finding container b283ec2bf84bb2c561641cfb5b7d5076833d9be72b69a716cc291680238afcb0: Status 404 returned error can't find the container with id b283ec2bf84bb2c561641cfb5b7d5076833d9be72b69a716cc291680238afcb0
Jul 07 00:03:40.090992 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.090953    2564 generic.go:356] "Generic (PLEG): container finished" podID="e35a51df-9478-4312-a105-c0802eed8b32" containerID="93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1" exitCode=0
Jul 07 00:03:40.091416 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.091009    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:40.091416 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.091068    2564 scope.go:117] "RemoveContainer" containerID="93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1"
Jul 07 00:03:40.093081 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.093045    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp" event={"ID":"8aacd94b-8b9d-4648-b88d-60a4d59c5f66","Type":"ContainerStarted","Data":"b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041"}
Jul 07 00:03:40.093206 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.093125    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:40.093206 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.093153    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp" event={"ID":"8aacd94b-8b9d-4648-b88d-60a4d59c5f66","Type":"ContainerStarted","Data":"0eae5a8abf2a193a0d4053ded0a4da14827d0c7ebeb8e62c346902e2c7039a87"}
Jul 07 00:03:40.094133 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:40.094107    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:40.094643 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.094625    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:40.096423 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.096397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p" event={"ID":"e64af115-de23-4f39-9bc6-eb09bdb125ca","Type":"ContainerStarted","Data":"930fc73e606bc35330b023aa4cc254b897ea47d7efd559103ba49c3e2b82363a"}
Jul 07 00:03:40.096604 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.096579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p" event={"ID":"e64af115-de23-4f39-9bc6-eb09bdb125ca","Type":"ContainerStarted","Data":"b283ec2bf84bb2c561641cfb5b7d5076833d9be72b69a716cc291680238afcb0"}
Jul 07 00:03:40.096705 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.096621    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:40.096705 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:40.096415    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="e35a51df-9478-4312-a105-c0802eed8b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:40.101027 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.101010    2564 scope.go:117] "RemoveContainer" containerID="93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1"
Jul 07 00:03:40.101280 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:40.101260    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1\": container with ID starting with 93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1 not found: ID does not exist" containerID="93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1"
Jul 07 00:03:40.101346 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.101287    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1"} err="failed to get container status \"93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1\": rpc error: code = NotFound desc = could not find container \"93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1\": container with ID starting with 93da1b9a03ffe98589dd53e77032b9da54b4517d3bea5f5e52af60e73d90e7c1 not found: ID does not exist"
Jul 07 00:03:40.101346 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.101302    2564 scope.go:117] "RemoveContainer" containerID="5019eff6457500a4c3ccde31b27cc79cf7c1ef267fdd3ad042a18635d440a146"
Jul 07 00:03:40.112668 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:40.112646    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m8l6l\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="e35a51df-9478-4312-a105-c0802eed8b32" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m8l6l"
Jul 07 00:03:40.114766 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.114748    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" path="/var/lib/kubelet/pods/bcb0a02b-a4bf-4835-b067-6f88fd7db071/volumes"
Jul 07 00:03:40.115045 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.115034    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e35a51df-9478-4312-a105-c0802eed8b32" path="/var/lib/kubelet/pods/e35a51df-9478-4312-a105-c0802eed8b32/volumes"
Jul 07 00:03:40.152356 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:40.152307    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p" podStartSLOduration=1.152290217 podStartE2EDuration="1.152290217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:40.151029067 +0000 UTC m=+452.564042992" watchObservedRunningTime="2026-07-07 00:03:40.152290217 +0000 UTC m=+452.565304124"
Jul 07 00:03:40.229075 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:03:40.229037    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jczv5\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="bcb0a02b-a4bf-4835-b067-6f88fd7db071" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jczv5"
Jul 07 00:03:51.106143 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:51.106117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:03:51.106633 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:51.106181    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx9p"
Jul 07 00:03:51.127824 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:03:51.127779    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp" podStartSLOduration=13.127765264 podStartE2EDuration="13.127765264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:40.2269172 +0000 UTC m=+452.639931108" watchObservedRunningTime="2026-07-07 00:03:51.127765264 +0000 UTC m=+463.540779172"
Jul 07 00:04:09.687600 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:09.687565    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"]
Jul 07 00:04:09.688006 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:09.687789    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp" podUID="8aacd94b-8b9d-4648-b88d-60a4d59c5f66" containerName="manager" containerID="cri-o://b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041" gracePeriod=10
Jul 07 00:04:09.933218 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:09.933192    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:04:10.081655 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.081575    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-extensions-socket-volume\") pod \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") "
Jul 07 00:04:10.081655 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.081616    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqf4v\" (UniqueName: \"kubernetes.io/projected/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-kube-api-access-qqf4v\") pod \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\" (UID: \"8aacd94b-8b9d-4648-b88d-60a4d59c5f66\") "
Jul 07 00:04:10.081929 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.081902    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8aacd94b-8b9d-4648-b88d-60a4d59c5f66" (UID: "8aacd94b-8b9d-4648-b88d-60a4d59c5f66"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:04:10.083676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.083652    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-kube-api-access-qqf4v" (OuterVolumeSpecName: "kube-api-access-qqf4v") pod "8aacd94b-8b9d-4648-b88d-60a4d59c5f66" (UID: "8aacd94b-8b9d-4648-b88d-60a4d59c5f66"). InnerVolumeSpecName "kube-api-access-qqf4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:10.182708 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.182680    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-extensions-socket-volume\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:10.182708 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.182704    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqf4v\" (UniqueName: \"kubernetes.io/projected/8aacd94b-8b9d-4648-b88d-60a4d59c5f66-kube-api-access-qqf4v\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:10.238537 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.238507    2564 generic.go:356] "Generic (PLEG): container finished" podID="8aacd94b-8b9d-4648-b88d-60a4d59c5f66" containerID="b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041" exitCode=0
Jul 07 00:04:10.238644 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.238552    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"
Jul 07 00:04:10.238644 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.238595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp" event={"ID":"8aacd94b-8b9d-4648-b88d-60a4d59c5f66","Type":"ContainerDied","Data":"b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041"}
Jul 07 00:04:10.238644 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.238633    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp" event={"ID":"8aacd94b-8b9d-4648-b88d-60a4d59c5f66","Type":"ContainerDied","Data":"0eae5a8abf2a193a0d4053ded0a4da14827d0c7ebeb8e62c346902e2c7039a87"}
Jul 07 00:04:10.238767 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.238654    2564 scope.go:117] "RemoveContainer" containerID="b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041"
Jul 07 00:04:10.246673 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.246651    2564 scope.go:117] "RemoveContainer" containerID="b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041"
Jul 07 00:04:10.246932 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:04:10.246905    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041\": container with ID starting with b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041 not found: ID does not exist" containerID="b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041"
Jul 07 00:04:10.246996 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.246934    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041"} err="failed to get container status \"b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041\": rpc error: code = NotFound desc = could not find container \"b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041\": container with ID starting with b30c938ad2cd4ab17038a863fed1935b3fa74124e60d820f8ec18756d78ab041 not found: ID does not exist"
Jul 07 00:04:10.266758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.266737    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"]
Jul 07 00:04:10.271090 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:10.271066    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h8fpp"]
Jul 07 00:04:12.114519 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:12.114489    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aacd94b-8b9d-4648-b88d-60a4d59c5f66" path="/var/lib/kubelet/pods/8aacd94b-8b9d-4648-b88d-60a4d59c5f66/volumes"
Jul 07 00:04:26.175787 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.175753    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"]
Jul 07 00:04:26.176258 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.176099    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aacd94b-8b9d-4648-b88d-60a4d59c5f66" containerName="manager"
Jul 07 00:04:26.176258 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.176115    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aacd94b-8b9d-4648-b88d-60a4d59c5f66" containerName="manager"
Jul 07 00:04:26.176358 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.176273    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aacd94b-8b9d-4648-b88d-60a4d59c5f66" containerName="manager"
Jul 07 00:04:26.180723 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.180705    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.183596 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.183577    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2vdrm\""
Jul 07 00:04:26.190995 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.190975    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"]
Jul 07 00:04:26.324237 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324237 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324238    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7143c880-61d2-423d-a37e-2cb66b2f630f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5sqk\" (UniqueName: \"kubernetes.io/projected/7143c880-61d2-423d-a37e-2cb66b2f630f-kube-api-access-b5sqk\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.324501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.324387    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425199 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7143c880-61d2-423d-a37e-2cb66b2f630f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425377 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5sqk\" (UniqueName: \"kubernetes.io/projected/7143c880-61d2-423d-a37e-2cb66b2f630f-kube-api-access-b5sqk\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425715    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.425761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425725    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.426044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425983    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.426044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.425989    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.426275 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.426257    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7143c880-61d2-423d-a37e-2cb66b2f630f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.427688 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.427665    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.427810 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.427791    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.434596 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.434573    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5sqk\" (UniqueName: \"kubernetes.io/projected/7143c880-61d2-423d-a37e-2cb66b2f630f-kube-api-access-b5sqk\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.434729 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.434644    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7143c880-61d2-423d-a37e-2cb66b2f630f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kf8gf\" (UID: \"7143c880-61d2-423d-a37e-2cb66b2f630f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.491680 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.491655    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:26.623102 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.623078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"]
Jul 07 00:04:26.625000 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:04:26.624972    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7143c880_61d2_423d_a37e_2cb66b2f630f.slice/crio-e26300f43f2b5212abfde75f025b389a3b56426bce5eb508a33b798d782422ee WatchSource:0}: Error finding container e26300f43f2b5212abfde75f025b389a3b56426bce5eb508a33b798d782422ee: Status 404 returned error can't find the container with id e26300f43f2b5212abfde75f025b389a3b56426bce5eb508a33b798d782422ee
Jul 07 00:04:26.627045 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.627009    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:04:26.627122 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.627089    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:04:26.627159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:26.627132    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:04:27.314523 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:27.314487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" event={"ID":"7143c880-61d2-423d-a37e-2cb66b2f630f","Type":"ContainerStarted","Data":"eeb2c71c1ca943b12a3e2ae7432bef21f2053cf930be27cd9e58224e49b720fa"}
Jul 07 00:04:27.314935 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:27.314545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" event={"ID":"7143c880-61d2-423d-a37e-2cb66b2f630f","Type":"ContainerStarted","Data":"e26300f43f2b5212abfde75f025b389a3b56426bce5eb508a33b798d782422ee"}
Jul 07 00:04:27.334601 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:27.334555    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" podStartSLOduration=1.334539408 podStartE2EDuration="1.334539408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:27.33309994 +0000 UTC m=+499.746113854" watchObservedRunningTime="2026-07-07 00:04:27.334539408 +0000 UTC m=+499.747553315"
Jul 07 00:04:27.492776 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:27.492749    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:27.497444 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:27.497422    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:28.319256 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.319222    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:28.320102 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.320083    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:04:28.337584 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.337554    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:28.341212 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.341195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.343987 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.343963    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4nstc\""
Jul 07 00:04:28.344319 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.344303    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 00:04:28.350500 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.350478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:28.434158 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.434122    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:28.441227 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.441194    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28cf\" (UniqueName: \"kubernetes.io/projected/e70813af-1641-4fa4-9c15-9b8829169d3a-kube-api-access-l28cf\") pod \"limitador-limitador-64c94fd64f-j6bqn\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.441470 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.441436    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e70813af-1641-4fa4-9c15-9b8829169d3a-config-file\") pod \"limitador-limitador-64c94fd64f-j6bqn\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.542841 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.542801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e70813af-1641-4fa4-9c15-9b8829169d3a-config-file\") pod \"limitador-limitador-64c94fd64f-j6bqn\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.543035 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.542949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28cf\" (UniqueName: \"kubernetes.io/projected/e70813af-1641-4fa4-9c15-9b8829169d3a-kube-api-access-l28cf\") pod \"limitador-limitador-64c94fd64f-j6bqn\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.543661 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.543635    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e70813af-1641-4fa4-9c15-9b8829169d3a-config-file\") pod \"limitador-limitador-64c94fd64f-j6bqn\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.552915 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.552885    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28cf\" (UniqueName: \"kubernetes.io/projected/e70813af-1641-4fa4-9c15-9b8829169d3a-kube-api-access-l28cf\") pod \"limitador-limitador-64c94fd64f-j6bqn\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.654625 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.654598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:28.785309 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:28.785287    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:28.787988 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:04:28.787960    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70813af_1641_4fa4_9c15_9b8829169d3a.slice/crio-8ac466e624b9d9e09bac8fcdda25b7641836d0525b418b19298a143ef2ba3122 WatchSource:0}: Error finding container 8ac466e624b9d9e09bac8fcdda25b7641836d0525b418b19298a143ef2ba3122: Status 404 returned error can't find the container with id 8ac466e624b9d9e09bac8fcdda25b7641836d0525b418b19298a143ef2ba3122
Jul 07 00:04:29.154768 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.154729    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-q9z2c"]
Jul 07 00:04:29.159278 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.159255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:29.161868 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.161851    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ztdh2\""
Jul 07 00:04:29.165430 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.165401    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q9z2c"]
Jul 07 00:04:29.248351 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.248308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56dxf\" (UniqueName: \"kubernetes.io/projected/d719a0f5-0a73-4765-9e15-2b4da473539f-kube-api-access-56dxf\") pod \"authorino-58cd955675-q9z2c\" (UID: \"d719a0f5-0a73-4765-9e15-2b4da473539f\") " pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:29.312201 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.312165    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-cmvnn"]
Jul 07 00:04:29.315513 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.315495    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:04:29.324127 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.324095    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cmvnn"]
Jul 07 00:04:29.326293 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.326261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn" event={"ID":"e70813af-1641-4fa4-9c15-9b8829169d3a","Type":"ContainerStarted","Data":"8ac466e624b9d9e09bac8fcdda25b7641836d0525b418b19298a143ef2ba3122"}
Jul 07 00:04:29.349191 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.349150    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6xz8\" (UniqueName: \"kubernetes.io/projected/a883ef0e-e3fc-47ef-8f8f-a2741a268000-kube-api-access-l6xz8\") pod \"authorino-75d556bd77-cmvnn\" (UID: \"a883ef0e-e3fc-47ef-8f8f-a2741a268000\") " pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:04:29.349387 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.349229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56dxf\" (UniqueName: \"kubernetes.io/projected/d719a0f5-0a73-4765-9e15-2b4da473539f-kube-api-access-56dxf\") pod \"authorino-58cd955675-q9z2c\" (UID: \"d719a0f5-0a73-4765-9e15-2b4da473539f\") " pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:29.357802 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.357775    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56dxf\" (UniqueName: \"kubernetes.io/projected/d719a0f5-0a73-4765-9e15-2b4da473539f-kube-api-access-56dxf\") pod \"authorino-58cd955675-q9z2c\" (UID: \"d719a0f5-0a73-4765-9e15-2b4da473539f\") " pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:29.449884 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.449789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6xz8\" (UniqueName: \"kubernetes.io/projected/a883ef0e-e3fc-47ef-8f8f-a2741a268000-kube-api-access-l6xz8\") pod \"authorino-75d556bd77-cmvnn\" (UID: \"a883ef0e-e3fc-47ef-8f8f-a2741a268000\") " pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:04:29.458303 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.458276    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6xz8\" (UniqueName: \"kubernetes.io/projected/a883ef0e-e3fc-47ef-8f8f-a2741a268000-kube-api-access-l6xz8\") pod \"authorino-75d556bd77-cmvnn\" (UID: \"a883ef0e-e3fc-47ef-8f8f-a2741a268000\") " pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:04:29.470068 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.470034    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:29.603726 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.603701    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q9z2c"]
Jul 07 00:04:29.605770 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:04:29.605742    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd719a0f5_0a73_4765_9e15_2b4da473539f.slice/crio-35724d6a089e994b2d01feced6c0aed3dba4ccb589b72f209a1f4e123947c9af WatchSource:0}: Error finding container 35724d6a089e994b2d01feced6c0aed3dba4ccb589b72f209a1f4e123947c9af: Status 404 returned error can't find the container with id 35724d6a089e994b2d01feced6c0aed3dba4ccb589b72f209a1f4e123947c9af
Jul 07 00:04:29.625311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.625287    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:04:29.794832 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:29.794801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cmvnn"]
Jul 07 00:04:29.797995 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:04:29.797966    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda883ef0e_e3fc_47ef_8f8f_a2741a268000.slice/crio-be4007a70be84be203dabfa26703e73d98ea150f7f75495fbd4040b30de33cd1 WatchSource:0}: Error finding container be4007a70be84be203dabfa26703e73d98ea150f7f75495fbd4040b30de33cd1: Status 404 returned error can't find the container with id be4007a70be84be203dabfa26703e73d98ea150f7f75495fbd4040b30de33cd1
Jul 07 00:04:30.337618 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:30.337576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cmvnn" event={"ID":"a883ef0e-e3fc-47ef-8f8f-a2741a268000","Type":"ContainerStarted","Data":"be4007a70be84be203dabfa26703e73d98ea150f7f75495fbd4040b30de33cd1"}
Jul 07 00:04:30.341507 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:30.341478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q9z2c" event={"ID":"d719a0f5-0a73-4765-9e15-2b4da473539f","Type":"ContainerStarted","Data":"35724d6a089e994b2d01feced6c0aed3dba4ccb589b72f209a1f4e123947c9af"}
Jul 07 00:04:34.370720 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.370682    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q9z2c" event={"ID":"d719a0f5-0a73-4765-9e15-2b4da473539f","Type":"ContainerStarted","Data":"4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6"}
Jul 07 00:04:34.372288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.372251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn" event={"ID":"e70813af-1641-4fa4-9c15-9b8829169d3a","Type":"ContainerStarted","Data":"be94186c1204c09a3634d9fa9280507c99591b7b135ba7cbdab54f51f15ead08"}
Jul 07 00:04:34.372399 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.372351    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:34.373566 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.373545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cmvnn" event={"ID":"a883ef0e-e3fc-47ef-8f8f-a2741a268000","Type":"ContainerStarted","Data":"4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297"}
Jul 07 00:04:34.394004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.393948    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-q9z2c" podStartSLOduration=0.969666333 podStartE2EDuration="5.393928045s" totalImagesPullingTime="4.424261712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:34.393663026 +0000 UTC m=+506.806676968" watchObservedRunningTime="2026-07-07 00:04:34.393928045 +0000 UTC m=+506.806941955"
Jul 07 00:04:34.415524 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.415431    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn" podStartSLOduration=1.12271541 podStartE2EDuration="6.415409802s" totalImagesPullingTime="5.292694392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:34.413307772 +0000 UTC m=+506.826321682" watchObservedRunningTime="2026-07-07 00:04:34.415409802 +0000 UTC m=+506.828423713"
Jul 07 00:04:34.430083 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.430033    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-cmvnn" podStartSLOduration=1.162987865 podStartE2EDuration="5.430015865s" totalImagesPullingTime="4.267028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:34.428259907 +0000 UTC m=+506.841273843" watchObservedRunningTime="2026-07-07 00:04:34.430015865 +0000 UTC m=+506.843029772"
Jul 07 00:04:34.455942 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:34.455908    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q9z2c"]
Jul 07 00:04:36.382981 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:36.382942    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-q9z2c" podUID="d719a0f5-0a73-4765-9e15-2b4da473539f" containerName="authorino" containerID="cri-o://4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6" gracePeriod=30
Jul 07 00:04:36.638139 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:36.638074    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:36.723218 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:36.723180    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56dxf\" (UniqueName: \"kubernetes.io/projected/d719a0f5-0a73-4765-9e15-2b4da473539f-kube-api-access-56dxf\") pod \"d719a0f5-0a73-4765-9e15-2b4da473539f\" (UID: \"d719a0f5-0a73-4765-9e15-2b4da473539f\") "
Jul 07 00:04:36.725241 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:36.725215    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d719a0f5-0a73-4765-9e15-2b4da473539f-kube-api-access-56dxf" (OuterVolumeSpecName: "kube-api-access-56dxf") pod "d719a0f5-0a73-4765-9e15-2b4da473539f" (UID: "d719a0f5-0a73-4765-9e15-2b4da473539f"). InnerVolumeSpecName "kube-api-access-56dxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:36.824157 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:36.824116    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56dxf\" (UniqueName: \"kubernetes.io/projected/d719a0f5-0a73-4765-9e15-2b4da473539f-kube-api-access-56dxf\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:37.388362 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.388322    2564 generic.go:356] "Generic (PLEG): container finished" podID="d719a0f5-0a73-4765-9e15-2b4da473539f" containerID="4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6" exitCode=0
Jul 07 00:04:37.388800 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.388384    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q9z2c"
Jul 07 00:04:37.388800 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.388404    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q9z2c" event={"ID":"d719a0f5-0a73-4765-9e15-2b4da473539f","Type":"ContainerDied","Data":"4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6"}
Jul 07 00:04:37.388800 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.388470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q9z2c" event={"ID":"d719a0f5-0a73-4765-9e15-2b4da473539f","Type":"ContainerDied","Data":"35724d6a089e994b2d01feced6c0aed3dba4ccb589b72f209a1f4e123947c9af"}
Jul 07 00:04:37.388800 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.388491    2564 scope.go:117] "RemoveContainer" containerID="4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6"
Jul 07 00:04:37.396963 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.396946    2564 scope.go:117] "RemoveContainer" containerID="4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6"
Jul 07 00:04:37.397220 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:04:37.397201    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6\": container with ID starting with 4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6 not found: ID does not exist" containerID="4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6"
Jul 07 00:04:37.397285 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.397233    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6"} err="failed to get container status \"4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6\": rpc error: code = NotFound desc = could not find container \"4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6\": container with ID starting with 4098c8d688a1940c20cd9177e06e4c6fe00f780c6a96accab515e3e312dfb6e6 not found: ID does not exist"
Jul 07 00:04:37.408193 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.408164    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q9z2c"]
Jul 07 00:04:37.411098 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:37.411075    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-q9z2c"]
Jul 07 00:04:38.114788 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:38.114757    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d719a0f5-0a73-4765-9e15-2b4da473539f" path="/var/lib/kubelet/pods/d719a0f5-0a73-4765-9e15-2b4da473539f/volumes"
Jul 07 00:04:42.937003 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:42.936966    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:42.937414 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:42.937206    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn" podUID="e70813af-1641-4fa4-9c15-9b8829169d3a" containerName="limitador" containerID="cri-o://be94186c1204c09a3634d9fa9280507c99591b7b135ba7cbdab54f51f15ead08" gracePeriod=30
Jul 07 00:04:42.939369 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:42.939348    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:43.424126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.424090    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70813af-1641-4fa4-9c15-9b8829169d3a" containerID="be94186c1204c09a3634d9fa9280507c99591b7b135ba7cbdab54f51f15ead08" exitCode=0
Jul 07 00:04:43.424346 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.424150    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn" event={"ID":"e70813af-1641-4fa4-9c15-9b8829169d3a","Type":"ContainerDied","Data":"be94186c1204c09a3634d9fa9280507c99591b7b135ba7cbdab54f51f15ead08"}
Jul 07 00:04:43.478381 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.478357    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:43.582398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.582313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e70813af-1641-4fa4-9c15-9b8829169d3a-config-file\") pod \"e70813af-1641-4fa4-9c15-9b8829169d3a\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") "
Jul 07 00:04:43.582574 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.582419    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l28cf\" (UniqueName: \"kubernetes.io/projected/e70813af-1641-4fa4-9c15-9b8829169d3a-kube-api-access-l28cf\") pod \"e70813af-1641-4fa4-9c15-9b8829169d3a\" (UID: \"e70813af-1641-4fa4-9c15-9b8829169d3a\") "
Jul 07 00:04:43.582712 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.582689    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70813af-1641-4fa4-9c15-9b8829169d3a-config-file" (OuterVolumeSpecName: "config-file") pod "e70813af-1641-4fa4-9c15-9b8829169d3a" (UID: "e70813af-1641-4fa4-9c15-9b8829169d3a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:04:43.584381 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.584358    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70813af-1641-4fa4-9c15-9b8829169d3a-kube-api-access-l28cf" (OuterVolumeSpecName: "kube-api-access-l28cf") pod "e70813af-1641-4fa4-9c15-9b8829169d3a" (UID: "e70813af-1641-4fa4-9c15-9b8829169d3a"). InnerVolumeSpecName "kube-api-access-l28cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:43.683744 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.683707    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l28cf\" (UniqueName: \"kubernetes.io/projected/e70813af-1641-4fa4-9c15-9b8829169d3a-kube-api-access-l28cf\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:43.683744 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:43.683739    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e70813af-1641-4fa4-9c15-9b8829169d3a-config-file\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:44.430725 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.430634    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"
Jul 07 00:04:44.430725 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.430693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j6bqn" event={"ID":"e70813af-1641-4fa4-9c15-9b8829169d3a","Type":"ContainerDied","Data":"8ac466e624b9d9e09bac8fcdda25b7641836d0525b418b19298a143ef2ba3122"}
Jul 07 00:04:44.431177 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.430748    2564 scope.go:117] "RemoveContainer" containerID="be94186c1204c09a3634d9fa9280507c99591b7b135ba7cbdab54f51f15ead08"
Jul 07 00:04:44.447846 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.447823    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:44.450589 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.450567    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j6bqn"]
Jul 07 00:04:44.882624 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.882590    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-mm8qm"]
Jul 07 00:04:44.883121 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.883105    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70813af-1641-4fa4-9c15-9b8829169d3a" containerName="limitador"
Jul 07 00:04:44.883215 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.883126    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70813af-1641-4fa4-9c15-9b8829169d3a" containerName="limitador"
Jul 07 00:04:44.883215 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.883145    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d719a0f5-0a73-4765-9e15-2b4da473539f" containerName="authorino"
Jul 07 00:04:44.883215 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.883154    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d719a0f5-0a73-4765-9e15-2b4da473539f" containerName="authorino"
Jul 07 00:04:44.883307 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.883274    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70813af-1641-4fa4-9c15-9b8829169d3a" containerName="limitador"
Jul 07 00:04:44.883307 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.883289    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d719a0f5-0a73-4765-9e15-2b4da473539f" containerName="authorino"
Jul 07 00:04:44.887843 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.887821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:44.891823 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.891567    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vrnjc\""
Jul 07 00:04:44.891823 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.891631    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 07 00:04:44.893500 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.893478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mm8qm"]
Jul 07 00:04:44.993780 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.993749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvqcb\" (UniqueName: \"kubernetes.io/projected/ac916e15-2868-41ec-92b6-9af3ce0d710d-kube-api-access-tvqcb\") pod \"postgres-69fb4594cf-mm8qm\" (UID: \"ac916e15-2868-41ec-92b6-9af3ce0d710d\") " pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:44.993948 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:44.993816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ac916e15-2868-41ec-92b6-9af3ce0d710d-data\") pod \"postgres-69fb4594cf-mm8qm\" (UID: \"ac916e15-2868-41ec-92b6-9af3ce0d710d\") " pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:45.095068 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.095034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ac916e15-2868-41ec-92b6-9af3ce0d710d-data\") pod \"postgres-69fb4594cf-mm8qm\" (UID: \"ac916e15-2868-41ec-92b6-9af3ce0d710d\") " pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:45.095236 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.095103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqcb\" (UniqueName: \"kubernetes.io/projected/ac916e15-2868-41ec-92b6-9af3ce0d710d-kube-api-access-tvqcb\") pod \"postgres-69fb4594cf-mm8qm\" (UID: \"ac916e15-2868-41ec-92b6-9af3ce0d710d\") " pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:45.095439 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.095421    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ac916e15-2868-41ec-92b6-9af3ce0d710d-data\") pod \"postgres-69fb4594cf-mm8qm\" (UID: \"ac916e15-2868-41ec-92b6-9af3ce0d710d\") " pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:45.104822 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.104798    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqcb\" (UniqueName: \"kubernetes.io/projected/ac916e15-2868-41ec-92b6-9af3ce0d710d-kube-api-access-tvqcb\") pod \"postgres-69fb4594cf-mm8qm\" (UID: \"ac916e15-2868-41ec-92b6-9af3ce0d710d\") " pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:45.199967 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.199890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:45.324108 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.324084    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mm8qm"]
Jul 07 00:04:45.326466 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:04:45.326427    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac916e15_2868_41ec_92b6_9af3ce0d710d.slice/crio-2df1e62c96f41100a9d29c29a2ab076062df5f2054e1d906a80ebaa9fbff4c9d WatchSource:0}: Error finding container 2df1e62c96f41100a9d29c29a2ab076062df5f2054e1d906a80ebaa9fbff4c9d: Status 404 returned error can't find the container with id 2df1e62c96f41100a9d29c29a2ab076062df5f2054e1d906a80ebaa9fbff4c9d
Jul 07 00:04:45.437745 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:45.437713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mm8qm" event={"ID":"ac916e15-2868-41ec-92b6-9af3ce0d710d","Type":"ContainerStarted","Data":"2df1e62c96f41100a9d29c29a2ab076062df5f2054e1d906a80ebaa9fbff4c9d"}
Jul 07 00:04:46.118591 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:46.118558    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70813af-1641-4fa4-9c15-9b8829169d3a" path="/var/lib/kubelet/pods/e70813af-1641-4fa4-9c15-9b8829169d3a/volumes"
Jul 07 00:04:50.467007 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:50.466975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mm8qm" event={"ID":"ac916e15-2868-41ec-92b6-9af3ce0d710d","Type":"ContainerStarted","Data":"054a34af5216bdb2323da3e9dbbffe313648201e84c54f2211273ef14fc5e1f5"}
Jul 07 00:04:50.467382 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:50.467039    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:50.482934 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:50.482887    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-mm8qm" podStartSLOduration=1.9090814379999999 podStartE2EDuration="6.482873183s" totalImagesPullingTime="4.573791745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:50.482339027 +0000 UTC m=+522.895352963" watchObservedRunningTime="2026-07-07 00:04:50.482873183 +0000 UTC m=+522.895887091"
Jul 07 00:04:56.502062 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:56.502036    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-mm8qm"
Jul 07 00:04:57.062244 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.062207    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc6g2"]
Jul 07 00:04:57.067037 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.067017    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:57.071868 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.071846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc6g2"]
Jul 07 00:04:57.204756 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.204725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrqdj\" (UniqueName: \"kubernetes.io/projected/cab87fbe-a68c-47d7-98a0-37dc9a714e4c-kube-api-access-xrqdj\") pod \"authorino-5d4b67c446-fc6g2\" (UID: \"cab87fbe-a68c-47d7-98a0-37dc9a714e4c\") " pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:57.268282 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.268248    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc6g2"]
Jul 07 00:04:57.268536 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:04:57.268518    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xrqdj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-fc6g2" podUID="cab87fbe-a68c-47d7-98a0-37dc9a714e4c"
Jul 07 00:04:57.291219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.291189    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-t8sqs"]
Jul 07 00:04:57.294527 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.294508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.297506 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.297484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 00:04:57.301503 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.301485    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-t8sqs"]
Jul 07 00:04:57.305221 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.305197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrqdj\" (UniqueName: \"kubernetes.io/projected/cab87fbe-a68c-47d7-98a0-37dc9a714e4c-kube-api-access-xrqdj\") pod \"authorino-5d4b67c446-fc6g2\" (UID: \"cab87fbe-a68c-47d7-98a0-37dc9a714e4c\") " pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:57.315074 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.315019    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrqdj\" (UniqueName: \"kubernetes.io/projected/cab87fbe-a68c-47d7-98a0-37dc9a714e4c-kube-api-access-xrqdj\") pod \"authorino-5d4b67c446-fc6g2\" (UID: \"cab87fbe-a68c-47d7-98a0-37dc9a714e4c\") " pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:57.336034 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.336007    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-t8sqs"]
Jul 07 00:04:57.336242 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:04:57.336223    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-g6fhm tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-g6fhm tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-t8sqs" podUID="d29874ae-26b6-4561-8ef6-3189b8c8f13d"
Jul 07 00:04:57.361618 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.361593    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54cffd5dc9-5gwd8"]
Jul 07 00:04:57.365087 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.365069    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.371224 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.371198    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54cffd5dc9-5gwd8"]
Jul 07 00:04:57.406188 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.406159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d29874ae-26b6-4561-8ef6-3189b8c8f13d-tls-cert\") pod \"authorino-76877994-t8sqs\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") " pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.406326 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.406246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6fhm\" (UniqueName: \"kubernetes.io/projected/d29874ae-26b6-4561-8ef6-3189b8c8f13d-kube-api-access-g6fhm\") pod \"authorino-76877994-t8sqs\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") " pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.499194 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.499164    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.499368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.499164    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:57.505920 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.505898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.506733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.506716    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfpg6\" (UniqueName: \"kubernetes.io/projected/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-kube-api-access-gfpg6\") pod \"authorino-54cffd5dc9-5gwd8\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") " pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.506835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.506789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d29874ae-26b6-4561-8ef6-3189b8c8f13d-tls-cert\") pod \"authorino-76877994-t8sqs\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") " pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.506835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.506815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-tls-cert\") pod \"authorino-54cffd5dc9-5gwd8\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") " pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.506943 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.506853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6fhm\" (UniqueName: \"kubernetes.io/projected/d29874ae-26b6-4561-8ef6-3189b8c8f13d-kube-api-access-g6fhm\") pod \"authorino-76877994-t8sqs\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") " pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.509155 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.509136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:57.509255 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.509142    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d29874ae-26b6-4561-8ef6-3189b8c8f13d-tls-cert\") pod \"authorino-76877994-t8sqs\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") " pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.520905 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.520886    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6fhm\" (UniqueName: \"kubernetes.io/projected/d29874ae-26b6-4561-8ef6-3189b8c8f13d-kube-api-access-g6fhm\") pod \"authorino-76877994-t8sqs\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") " pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:57.607812 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.607789    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6fhm\" (UniqueName: \"kubernetes.io/projected/d29874ae-26b6-4561-8ef6-3189b8c8f13d-kube-api-access-g6fhm\") pod \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") "
Jul 07 00:04:57.607941 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.607838    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d29874ae-26b6-4561-8ef6-3189b8c8f13d-tls-cert\") pod \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\" (UID: \"d29874ae-26b6-4561-8ef6-3189b8c8f13d\") "
Jul 07 00:04:57.607941 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.607876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrqdj\" (UniqueName: \"kubernetes.io/projected/cab87fbe-a68c-47d7-98a0-37dc9a714e4c-kube-api-access-xrqdj\") pod \"cab87fbe-a68c-47d7-98a0-37dc9a714e4c\" (UID: \"cab87fbe-a68c-47d7-98a0-37dc9a714e4c\") "
Jul 07 00:04:57.608065 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.608049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-tls-cert\") pod \"authorino-54cffd5dc9-5gwd8\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") " pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.608153 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.608129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfpg6\" (UniqueName: \"kubernetes.io/projected/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-kube-api-access-gfpg6\") pod \"authorino-54cffd5dc9-5gwd8\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") " pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.609949 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.609912    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29874ae-26b6-4561-8ef6-3189b8c8f13d-kube-api-access-g6fhm" (OuterVolumeSpecName: "kube-api-access-g6fhm") pod "d29874ae-26b6-4561-8ef6-3189b8c8f13d" (UID: "d29874ae-26b6-4561-8ef6-3189b8c8f13d"). InnerVolumeSpecName "kube-api-access-g6fhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:57.610036 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.609951    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab87fbe-a68c-47d7-98a0-37dc9a714e4c-kube-api-access-xrqdj" (OuterVolumeSpecName: "kube-api-access-xrqdj") pod "cab87fbe-a68c-47d7-98a0-37dc9a714e4c" (UID: "cab87fbe-a68c-47d7-98a0-37dc9a714e4c"). InnerVolumeSpecName "kube-api-access-xrqdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:57.610228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.610205    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d29874ae-26b6-4561-8ef6-3189b8c8f13d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d29874ae-26b6-4561-8ef6-3189b8c8f13d" (UID: "d29874ae-26b6-4561-8ef6-3189b8c8f13d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:04:57.610628 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.610611    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-tls-cert\") pod \"authorino-54cffd5dc9-5gwd8\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") " pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.616844 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.616818    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfpg6\" (UniqueName: \"kubernetes.io/projected/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-kube-api-access-gfpg6\") pod \"authorino-54cffd5dc9-5gwd8\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") " pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.675230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.675203    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:04:57.709827 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.709797    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrqdj\" (UniqueName: \"kubernetes.io/projected/cab87fbe-a68c-47d7-98a0-37dc9a714e4c-kube-api-access-xrqdj\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:57.709944 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.709830    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6fhm\" (UniqueName: \"kubernetes.io/projected/d29874ae-26b6-4561-8ef6-3189b8c8f13d-kube-api-access-g6fhm\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:57.709944 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.709852    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d29874ae-26b6-4561-8ef6-3189b8c8f13d-tls-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:04:57.802388 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:57.802355    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54cffd5dc9-5gwd8"]
Jul 07 00:04:57.805339 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:04:57.805311    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6bf9f27_83e4_4f71_9749_d8f9500c3f6f.slice/crio-f839f185582ae120060eac8d74b6753cf41d6378c5742a63a7c4f9bab172b748 WatchSource:0}: Error finding container f839f185582ae120060eac8d74b6753cf41d6378c5742a63a7c4f9bab172b748: Status 404 returned error can't find the container with id f839f185582ae120060eac8d74b6753cf41d6378c5742a63a7c4f9bab172b748
Jul 07 00:04:58.505078 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.504983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8" event={"ID":"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f","Type":"ContainerStarted","Data":"f839f185582ae120060eac8d74b6753cf41d6378c5742a63a7c4f9bab172b748"}
Jul 07 00:04:58.505078 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.505013    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-t8sqs"
Jul 07 00:04:58.505078 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.504993    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fc6g2"
Jul 07 00:04:58.532049 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.532024    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-t8sqs"]
Jul 07 00:04:58.533758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.533732    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-t8sqs"]
Jul 07 00:04:58.551172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.551147    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc6g2"]
Jul 07 00:04:58.554475 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:58.554438    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fc6g2"]
Jul 07 00:04:59.513264 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.513231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8" event={"ID":"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f","Type":"ContainerStarted","Data":"3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a"}
Jul 07 00:04:59.530207 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.530162    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8" podStartSLOduration=0.939629378 podStartE2EDuration="2.530146735s" totalImagesPullingTime="1.590517357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:59.52859065 +0000 UTC m=+531.941604579" watchObservedRunningTime="2026-07-07 00:04:59.530146735 +0000 UTC m=+531.943160643"
Jul 07 00:04:59.550914 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.550876    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cmvnn"]
Jul 07 00:04:59.551350 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.551077    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-cmvnn" podUID="a883ef0e-e3fc-47ef-8f8f-a2741a268000" containerName="authorino" containerID="cri-o://4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297" gracePeriod=30
Jul 07 00:04:59.815144 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.815119    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:04:59.932184 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.932107    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6xz8\" (UniqueName: \"kubernetes.io/projected/a883ef0e-e3fc-47ef-8f8f-a2741a268000-kube-api-access-l6xz8\") pod \"a883ef0e-e3fc-47ef-8f8f-a2741a268000\" (UID: \"a883ef0e-e3fc-47ef-8f8f-a2741a268000\") "
Jul 07 00:04:59.933869 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:04:59.933845    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a883ef0e-e3fc-47ef-8f8f-a2741a268000-kube-api-access-l6xz8" (OuterVolumeSpecName: "kube-api-access-l6xz8") pod "a883ef0e-e3fc-47ef-8f8f-a2741a268000" (UID: "a883ef0e-e3fc-47ef-8f8f-a2741a268000"). InnerVolumeSpecName "kube-api-access-l6xz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:05:00.033799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.033762    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6xz8\" (UniqueName: \"kubernetes.io/projected/a883ef0e-e3fc-47ef-8f8f-a2741a268000-kube-api-access-l6xz8\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:05:00.114610 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.114572    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cab87fbe-a68c-47d7-98a0-37dc9a714e4c" path="/var/lib/kubelet/pods/cab87fbe-a68c-47d7-98a0-37dc9a714e4c/volumes"
Jul 07 00:05:00.114866 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.114853    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d29874ae-26b6-4561-8ef6-3189b8c8f13d" path="/var/lib/kubelet/pods/d29874ae-26b6-4561-8ef6-3189b8c8f13d/volumes"
Jul 07 00:05:00.519036 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.519003    2564 generic.go:356] "Generic (PLEG): container finished" podID="a883ef0e-e3fc-47ef-8f8f-a2741a268000" containerID="4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297" exitCode=0
Jul 07 00:05:00.519230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.519046    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cmvnn"
Jul 07 00:05:00.519230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.519092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cmvnn" event={"ID":"a883ef0e-e3fc-47ef-8f8f-a2741a268000","Type":"ContainerDied","Data":"4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297"}
Jul 07 00:05:00.519230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.519134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cmvnn" event={"ID":"a883ef0e-e3fc-47ef-8f8f-a2741a268000","Type":"ContainerDied","Data":"be4007a70be84be203dabfa26703e73d98ea150f7f75495fbd4040b30de33cd1"}
Jul 07 00:05:00.519230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.519158    2564 scope.go:117] "RemoveContainer" containerID="4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297"
Jul 07 00:05:00.527828 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.527809    2564 scope.go:117] "RemoveContainer" containerID="4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297"
Jul 07 00:05:00.528132 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:05:00.528109    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297\": container with ID starting with 4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297 not found: ID does not exist" containerID="4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297"
Jul 07 00:05:00.528181 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.528140    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297"} err="failed to get container status \"4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297\": rpc error: code = NotFound desc = could not find container \"4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297\": container with ID starting with 4f6b29d8e262881a24f37da97cdc781cad2dd1ba810dd50fc8a50ddc25323297 not found: ID does not exist"
Jul 07 00:05:00.534425 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.534387    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cmvnn"]
Jul 07 00:05:00.535884 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:00.535864    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cmvnn"]
Jul 07 00:05:02.114926 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:02.114882    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a883ef0e-e3fc-47ef-8f8f-a2741a268000" path="/var/lib/kubelet/pods/a883ef0e-e3fc-47ef-8f8f-a2741a268000/volumes"
Jul 07 00:05:03.899905 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:03.899875    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79f8fd8d67-bdm2d"]
Jul 07 00:05:03.900345 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:03.900221    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a883ef0e-e3fc-47ef-8f8f-a2741a268000" containerName="authorino"
Jul 07 00:05:03.900345 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:03.900232    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a883ef0e-e3fc-47ef-8f8f-a2741a268000" containerName="authorino"
Jul 07 00:05:03.900345 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:03.900295    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a883ef0e-e3fc-47ef-8f8f-a2741a268000" containerName="authorino"
Jul 07 00:05:04.032098 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.032063    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79f8fd8d67-bdm2d"]
Jul 07 00:05:04.032253 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.032154    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.035007 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.034956    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 00:05:04.035007 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.035012    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 00:05:04.036083 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.036063    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-98rhj\""
Jul 07 00:05:04.077584 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.077546    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6966b9487-nr5cj"]
Jul 07 00:05:04.096349 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.096316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.097041 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.097019    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6966b9487-nr5cj"]
Jul 07 00:05:04.169286 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.169219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt7l4\" (UniqueName: \"kubernetes.io/projected/49b9cebe-6ead-4db7-9f47-7f3b1908e207-kube-api-access-mt7l4\") pod \"maas-controller-79f8fd8d67-bdm2d\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") " pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.169286 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.169272    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49b9cebe-6ead-4db7-9f47-7f3b1908e207-webhook-cert\") pod \"maas-controller-79f8fd8d67-bdm2d\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") " pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.269832 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.269800    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c43d94e-50e3-4c79-8b3e-3a538169c15a-webhook-cert\") pod \"maas-controller-6966b9487-nr5cj\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") " pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.269989 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.269838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7l4\" (UniqueName: \"kubernetes.io/projected/49b9cebe-6ead-4db7-9f47-7f3b1908e207-kube-api-access-mt7l4\") pod \"maas-controller-79f8fd8d67-bdm2d\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") " pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.269989 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.269897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49b9cebe-6ead-4db7-9f47-7f3b1908e207-webhook-cert\") pod \"maas-controller-79f8fd8d67-bdm2d\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") " pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.269989 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.269933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z5qm\" (UniqueName: \"kubernetes.io/projected/5c43d94e-50e3-4c79-8b3e-3a538169c15a-kube-api-access-5z5qm\") pod \"maas-controller-6966b9487-nr5cj\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") " pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.272388 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.272361    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49b9cebe-6ead-4db7-9f47-7f3b1908e207-webhook-cert\") pod \"maas-controller-79f8fd8d67-bdm2d\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") " pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.278880 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.278857    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7l4\" (UniqueName: \"kubernetes.io/projected/49b9cebe-6ead-4db7-9f47-7f3b1908e207-kube-api-access-mt7l4\") pod \"maas-controller-79f8fd8d67-bdm2d\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") " pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.342425 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.342399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:04.370532 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.370509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z5qm\" (UniqueName: \"kubernetes.io/projected/5c43d94e-50e3-4c79-8b3e-3a538169c15a-kube-api-access-5z5qm\") pod \"maas-controller-6966b9487-nr5cj\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") " pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.370667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.370589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c43d94e-50e3-4c79-8b3e-3a538169c15a-webhook-cert\") pod \"maas-controller-6966b9487-nr5cj\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") " pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.372742 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.372723    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c43d94e-50e3-4c79-8b3e-3a538169c15a-webhook-cert\") pod \"maas-controller-6966b9487-nr5cj\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") " pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.379406 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.379380    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z5qm\" (UniqueName: \"kubernetes.io/projected/5c43d94e-50e3-4c79-8b3e-3a538169c15a-kube-api-access-5z5qm\") pod \"maas-controller-6966b9487-nr5cj\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") " pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.406976 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.406947    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:04.488644 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.488605    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79f8fd8d67-bdm2d"]
Jul 07 00:05:04.492485 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:05:04.492429    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b9cebe_6ead_4db7_9f47_7f3b1908e207.slice/crio-9a11eabf85ef14aeae86c01479895d6a9f9e5d6740b476e6f2d670b6acbf43ad WatchSource:0}: Error finding container 9a11eabf85ef14aeae86c01479895d6a9f9e5d6740b476e6f2d670b6acbf43ad: Status 404 returned error can't find the container with id 9a11eabf85ef14aeae86c01479895d6a9f9e5d6740b476e6f2d670b6acbf43ad
Jul 07 00:05:04.542105 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.542077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d" event={"ID":"49b9cebe-6ead-4db7-9f47-7f3b1908e207","Type":"ContainerStarted","Data":"9a11eabf85ef14aeae86c01479895d6a9f9e5d6740b476e6f2d670b6acbf43ad"}
Jul 07 00:05:04.592285 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:04.592263    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6966b9487-nr5cj"]
Jul 07 00:05:04.594730 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:05:04.594707    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c43d94e_50e3_4c79_8b3e_3a538169c15a.slice/crio-94be91d7061206de133ff32868ebe0d3d76f3dbc7a5e4d9d7f482895e9ad4679 WatchSource:0}: Error finding container 94be91d7061206de133ff32868ebe0d3d76f3dbc7a5e4d9d7f482895e9ad4679: Status 404 returned error can't find the container with id 94be91d7061206de133ff32868ebe0d3d76f3dbc7a5e4d9d7f482895e9ad4679
Jul 07 00:05:05.553004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:05.552969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6966b9487-nr5cj" event={"ID":"5c43d94e-50e3-4c79-8b3e-3a538169c15a","Type":"ContainerStarted","Data":"94be91d7061206de133ff32868ebe0d3d76f3dbc7a5e4d9d7f482895e9ad4679"}
Jul 07 00:05:07.563235 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:07.563202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6966b9487-nr5cj" event={"ID":"5c43d94e-50e3-4c79-8b3e-3a538169c15a","Type":"ContainerStarted","Data":"8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6"}
Jul 07 00:05:07.563699 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:07.563312    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:07.564809 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:07.564788    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d" event={"ID":"49b9cebe-6ead-4db7-9f47-7f3b1908e207","Type":"ContainerStarted","Data":"40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817"}
Jul 07 00:05:07.564921 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:07.564834    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:07.586182 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:07.586106    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6966b9487-nr5cj" podStartSLOduration=0.826123974 podStartE2EDuration="3.586094088s" totalImagesPullingTime="2.759970114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:07.584279969 +0000 UTC m=+539.997293891" watchObservedRunningTime="2026-07-07 00:05:07.586094088 +0000 UTC m=+539.999107995"
Jul 07 00:05:07.606310 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:07.606269    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d" podStartSLOduration=1.754202583 podStartE2EDuration="4.606257273s" totalImagesPullingTime="2.85205469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:07.604808266 +0000 UTC m=+540.017822176" watchObservedRunningTime="2026-07-07 00:05:07.606257273 +0000 UTC m=+540.019271182"
Jul 07 00:05:18.574338 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.574306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:05:18.574812 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.574470    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:18.632227 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.632195    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79f8fd8d67-bdm2d"]
Jul 07 00:05:18.632447 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.632406    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d" podUID="49b9cebe-6ead-4db7-9f47-7f3b1908e207" containerName="manager" containerID="cri-o://40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817" gracePeriod=30
Jul 07 00:05:18.882925 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.882903    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:18.894926 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.894897    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49b9cebe-6ead-4db7-9f47-7f3b1908e207-webhook-cert\") pod \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") "
Jul 07 00:05:18.895026 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.894954    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt7l4\" (UniqueName: \"kubernetes.io/projected/49b9cebe-6ead-4db7-9f47-7f3b1908e207-kube-api-access-mt7l4\") pod \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\" (UID: \"49b9cebe-6ead-4db7-9f47-7f3b1908e207\") "
Jul 07 00:05:18.896982 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.896952    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49b9cebe-6ead-4db7-9f47-7f3b1908e207-kube-api-access-mt7l4" (OuterVolumeSpecName: "kube-api-access-mt7l4") pod "49b9cebe-6ead-4db7-9f47-7f3b1908e207" (UID: "49b9cebe-6ead-4db7-9f47-7f3b1908e207"). InnerVolumeSpecName "kube-api-access-mt7l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:05:18.897069 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.896993    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49b9cebe-6ead-4db7-9f47-7f3b1908e207-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "49b9cebe-6ead-4db7-9f47-7f3b1908e207" (UID: "49b9cebe-6ead-4db7-9f47-7f3b1908e207"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:05:18.996284 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.996252    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49b9cebe-6ead-4db7-9f47-7f3b1908e207-webhook-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:05:18.996284 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:18.996282    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt7l4\" (UniqueName: \"kubernetes.io/projected/49b9cebe-6ead-4db7-9f47-7f3b1908e207-kube-api-access-mt7l4\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:05:19.623018 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.622984    2564 generic.go:356] "Generic (PLEG): container finished" podID="49b9cebe-6ead-4db7-9f47-7f3b1908e207" containerID="40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817" exitCode=0
Jul 07 00:05:19.623505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.623030    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d"
Jul 07 00:05:19.623505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.623052    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d" event={"ID":"49b9cebe-6ead-4db7-9f47-7f3b1908e207","Type":"ContainerDied","Data":"40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817"}
Jul 07 00:05:19.623505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.623093    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79f8fd8d67-bdm2d" event={"ID":"49b9cebe-6ead-4db7-9f47-7f3b1908e207","Type":"ContainerDied","Data":"9a11eabf85ef14aeae86c01479895d6a9f9e5d6740b476e6f2d670b6acbf43ad"}
Jul 07 00:05:19.623505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.623112    2564 scope.go:117] "RemoveContainer" containerID="40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817"
Jul 07 00:05:19.635751 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.635727    2564 scope.go:117] "RemoveContainer" containerID="40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817"
Jul 07 00:05:19.636049 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:05:19.636026    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817\": container with ID starting with 40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817 not found: ID does not exist" containerID="40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817"
Jul 07 00:05:19.636126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.636056    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817"} err="failed to get container status \"40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817\": rpc error: code = NotFound desc = could not find container \"40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817\": container with ID starting with 40cb9e9e2b94ca55084dfed361ce16119200ec400293071fb2eeb9074d435817 not found: ID does not exist"
Jul 07 00:05:19.649568 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.649547    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79f8fd8d67-bdm2d"]
Jul 07 00:05:19.653862 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:19.653839    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79f8fd8d67-bdm2d"]
Jul 07 00:05:20.115740 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:20.115710    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49b9cebe-6ead-4db7-9f47-7f3b1908e207" path="/var/lib/kubelet/pods/49b9cebe-6ead-4db7-9f47-7f3b1908e207/volumes"
Jul 07 00:05:39.604538 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.604501    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-9bd647d7b-l86f6"]
Jul 07 00:05:39.604973 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.604854    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49b9cebe-6ead-4db7-9f47-7f3b1908e207" containerName="manager"
Jul 07 00:05:39.604973 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.604866    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="49b9cebe-6ead-4db7-9f47-7f3b1908e207" containerName="manager"
Jul 07 00:05:39.604973 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.604927    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="49b9cebe-6ead-4db7-9f47-7f3b1908e207" containerName="manager"
Jul 07 00:05:39.610421 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.610405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.615739 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.615714    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 07 00:05:39.616877 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.616780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-g89gr\""
Jul 07 00:05:39.633784 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.633753    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-9bd647d7b-l86f6"]
Jul 07 00:05:39.668731 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.668693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbww4\" (UniqueName: \"kubernetes.io/projected/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-kube-api-access-tbww4\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.668882 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.668775    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-openshift-service-ca\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.668882 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.668845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-maas-api-tls\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.770107 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.770076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-maas-api-tls\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.770267 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.770111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbww4\" (UniqueName: \"kubernetes.io/projected/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-kube-api-access-tbww4\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.770267 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.770149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-openshift-service-ca\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.770267 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:05:39.770223    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 07 00:05:39.770401 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:05:39.770295    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-maas-api-tls podName:5eb1b25c-e46d-4f2c-9332-38a6c55e5a36 nodeName:}" failed. No retries permitted until 2026-07-07 00:05:40.270278882 +0000 UTC m=+572.683292769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-maas-api-tls") pod "maas-api-9bd647d7b-l86f6" (UID: "5eb1b25c-e46d-4f2c-9332-38a6c55e5a36") : secret "maas-api-serving-cert" not found
Jul 07 00:05:39.770739 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.770722    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-openshift-service-ca\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:39.780621 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:39.780597    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbww4\" (UniqueName: \"kubernetes.io/projected/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-kube-api-access-tbww4\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:40.199350 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.199311    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-kd7mx"]
Jul 07 00:05:40.202740 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.202715    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.206772 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.206748    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-krw72\""
Jul 07 00:05:40.206915 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.206845    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 00:05:40.217221 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.217176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-kd7mx"]
Jul 07 00:05:40.275218 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.275181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4d2988-31b5-4bb6-b71f-77554ff8c10e-plugins-config-volume\") pod \"payload-processing-58f86c7798-kd7mx\" (UID: \"ea4d2988-31b5-4bb6-b71f-77554ff8c10e\") " pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.275435 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.275262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-maas-api-tls\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:40.275435 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.275315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdrkj\" (UniqueName: \"kubernetes.io/projected/ea4d2988-31b5-4bb6-b71f-77554ff8c10e-kube-api-access-wdrkj\") pod \"payload-processing-58f86c7798-kd7mx\" (UID: \"ea4d2988-31b5-4bb6-b71f-77554ff8c10e\") " pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.277906 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.277872    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb1b25c-e46d-4f2c-9332-38a6c55e5a36-maas-api-tls\") pod \"maas-api-9bd647d7b-l86f6\" (UID: \"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36\") " pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:40.375922 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.375891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4d2988-31b5-4bb6-b71f-77554ff8c10e-plugins-config-volume\") pod \"payload-processing-58f86c7798-kd7mx\" (UID: \"ea4d2988-31b5-4bb6-b71f-77554ff8c10e\") " pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.376087 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.376025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdrkj\" (UniqueName: \"kubernetes.io/projected/ea4d2988-31b5-4bb6-b71f-77554ff8c10e-kube-api-access-wdrkj\") pod \"payload-processing-58f86c7798-kd7mx\" (UID: \"ea4d2988-31b5-4bb6-b71f-77554ff8c10e\") " pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.376576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.376560    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4d2988-31b5-4bb6-b71f-77554ff8c10e-plugins-config-volume\") pod \"payload-processing-58f86c7798-kd7mx\" (UID: \"ea4d2988-31b5-4bb6-b71f-77554ff8c10e\") " pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.403020 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.402993    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdrkj\" (UniqueName: \"kubernetes.io/projected/ea4d2988-31b5-4bb6-b71f-77554ff8c10e-kube-api-access-wdrkj\") pod \"payload-processing-58f86c7798-kd7mx\" (UID: \"ea4d2988-31b5-4bb6-b71f-77554ff8c10e\") " pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.513586 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.513512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:40.520326 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.520308    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:40.699919 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.699885    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-kd7mx"]
Jul 07 00:05:40.701308 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:05:40.701278    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4d2988_31b5_4bb6_b71f_77554ff8c10e.slice/crio-23c91eace24c8f55576ed0884045ee2d2ab8b2a6565383a4cf14d8f83303f280 WatchSource:0}: Error finding container 23c91eace24c8f55576ed0884045ee2d2ab8b2a6565383a4cf14d8f83303f280: Status 404 returned error can't find the container with id 23c91eace24c8f55576ed0884045ee2d2ab8b2a6565383a4cf14d8f83303f280
Jul 07 00:05:40.721968 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.721947    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-9bd647d7b-l86f6"]
Jul 07 00:05:40.722274 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:40.722251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx" event={"ID":"ea4d2988-31b5-4bb6-b71f-77554ff8c10e","Type":"ContainerStarted","Data":"23c91eace24c8f55576ed0884045ee2d2ab8b2a6565383a4cf14d8f83303f280"}
Jul 07 00:05:40.723270 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:05:40.723251    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb1b25c_e46d_4f2c_9332_38a6c55e5a36.slice/crio-2f790b2ab07271d1307eb9ef780a7455389d44c880dd3708f9c859b2c25e7261 WatchSource:0}: Error finding container 2f790b2ab07271d1307eb9ef780a7455389d44c880dd3708f9c859b2c25e7261: Status 404 returned error can't find the container with id 2f790b2ab07271d1307eb9ef780a7455389d44c880dd3708f9c859b2c25e7261
Jul 07 00:05:41.755148 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:41.755112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-9bd647d7b-l86f6" event={"ID":"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36","Type":"ContainerStarted","Data":"2f790b2ab07271d1307eb9ef780a7455389d44c880dd3708f9c859b2c25e7261"}
Jul 07 00:05:42.763580 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:42.763533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx" event={"ID":"ea4d2988-31b5-4bb6-b71f-77554ff8c10e","Type":"ContainerStarted","Data":"d810291a7a72a3e8a03dd4dda789296105c14a7ad4309ecb0f6894b9a6d8791e"}
Jul 07 00:05:42.764045 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:42.763699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:42.791022 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:42.790970    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx" podStartSLOduration=1.217235003 podStartE2EDuration="2.790949401s" totalImagesPullingTime="1.573714398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:42.790614934 +0000 UTC m=+575.203628843" watchObservedRunningTime="2026-07-07 00:05:42.790949401 +0000 UTC m=+575.203963315"
Jul 07 00:05:43.770024 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:43.769984    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-9bd647d7b-l86f6" event={"ID":"5eb1b25c-e46d-4f2c-9332-38a6c55e5a36","Type":"ContainerStarted","Data":"081e61c2ae77ef710e66dc9e4c2e69bdd65b0241fcd6b108cf9199e4e266e369"}
Jul 07 00:05:43.770446 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:43.770230    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:43.788004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:43.787956    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-9bd647d7b-l86f6" podStartSLOduration=2.410883552 podStartE2EDuration="4.787942339s" totalImagesPullingTime="2.377058787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:43.7872626 +0000 UTC m=+576.200276510" watchObservedRunningTime="2026-07-07 00:05:43.787942339 +0000 UTC m=+576.200956246"
Jul 07 00:05:47.794228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:47.794133    2564 generic.go:356] "Generic (PLEG): container finished" podID="7143c880-61d2-423d-a37e-2cb66b2f630f" containerID="eeb2c71c1ca943b12a3e2ae7432bef21f2053cf930be27cd9e58224e49b720fa" exitCode=0
Jul 07 00:05:47.794228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:47.794182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" event={"ID":"7143c880-61d2-423d-a37e-2cb66b2f630f","Type":"ContainerDied","Data":"eeb2c71c1ca943b12a3e2ae7432bef21f2053cf930be27cd9e58224e49b720fa"}
Jul 07 00:05:47.794684 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:47.794599    2564 scope.go:117] "RemoveContainer" containerID="eeb2c71c1ca943b12a3e2ae7432bef21f2053cf930be27cd9e58224e49b720fa"
Jul 07 00:05:47.795660 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:47.795634    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:05:47.795726 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:47.795707    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:05:47.795762 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:47.795743    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:05:48.800256 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:48.800220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" event={"ID":"7143c880-61d2-423d-a37e-2cb66b2f630f","Type":"ContainerStarted","Data":"1d19bfc2e075c14621a0f39e69af52d1d1831f2d3df8bda6536ed3f834a418f9"}
Jul 07 00:05:49.492638 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:49.492598    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:05:49.492849 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:49.492830    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kf8gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 00:05:49.492903 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:49.492876    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" podUID="7143c880-61d2-423d-a37e-2cb66b2f630f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 00:05:49.784327 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:49.784257    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-9bd647d7b-l86f6"
Jul 07 00:05:50.492665 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:50.492627    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kf8gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 00:05:50.493059 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:50.492690    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" podUID="7143c880-61d2-423d-a37e-2cb66b2f630f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 00:05:51.492469 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:51.492415    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kf8gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 00:05:51.492631 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:51.492511    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" podUID="7143c880-61d2-423d-a37e-2cb66b2f630f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 00:05:52.492159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:52.492121    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kf8gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 00:05:52.492568 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:52.492189    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf" podUID="7143c880-61d2-423d-a37e-2cb66b2f630f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 00:05:53.495758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:53.495731    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:05:53.496119 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:53.495998    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:05:53.496705 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:53.496685    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kf8gf"
Jul 07 00:05:53.772556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:53.772444    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-kd7mx"
Jul 07 00:05:58.711072 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.711041    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"]
Jul 07 00:05:58.714812 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.714794    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.726063 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.726033    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 00:05:58.726063 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.726053    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 07 00:05:58.726267 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.726058    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 00:05:58.726267 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.726122    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bvnll\""
Jul 07 00:05:58.733475 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.733436    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"]
Jul 07 00:05:58.843501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.843432    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.843501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.843504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glcg\" (UniqueName: \"kubernetes.io/projected/b3ef597d-b888-4ddb-8c64-31316a83bfd1-kube-api-access-4glcg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.843747 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.843531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.843747 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.843588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.843747 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.843621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3ef597d-b888-4ddb-8c64-31316a83bfd1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.843747 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.843648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.944834 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.944796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945024 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.944845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4glcg\" (UniqueName: \"kubernetes.io/projected/b3ef597d-b888-4ddb-8c64-31316a83bfd1-kube-api-access-4glcg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945024 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.944881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945024 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.944925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945024 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.944954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3ef597d-b888-4ddb-8c64-31316a83bfd1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945389 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.945311    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945389 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.945324    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945389 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.945356    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.945389 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.945539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.947999 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.947975    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3ef597d-b888-4ddb-8c64-31316a83bfd1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.947999 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.947992    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3ef597d-b888-4ddb-8c64-31316a83bfd1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:58.954477 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:58.954398    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glcg\" (UniqueName: \"kubernetes.io/projected/b3ef597d-b888-4ddb-8c64-31316a83bfd1-kube-api-access-4glcg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n\" (UID: \"b3ef597d-b888-4ddb-8c64-31316a83bfd1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:59.025443 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:59.025349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:05:59.164936 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:59.164842    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"]
Jul 07 00:05:59.167997 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:05:59.167959    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ef597d_b888_4ddb_8c64_31316a83bfd1.slice/crio-5e4b4a8cd3f8ef4eed8840d962e49de15de4d9eb806826ba7e2b9b903f939881 WatchSource:0}: Error finding container 5e4b4a8cd3f8ef4eed8840d962e49de15de4d9eb806826ba7e2b9b903f939881: Status 404 returned error can't find the container with id 5e4b4a8cd3f8ef4eed8840d962e49de15de4d9eb806826ba7e2b9b903f939881
Jul 07 00:05:59.854126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:05:59.854084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n" event={"ID":"b3ef597d-b888-4ddb-8c64-31316a83bfd1","Type":"ContainerStarted","Data":"5e4b4a8cd3f8ef4eed8840d962e49de15de4d9eb806826ba7e2b9b903f939881"}
Jul 07 00:06:00.860368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:00.860330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n" event={"ID":"b3ef597d-b888-4ddb-8c64-31316a83bfd1","Type":"ContainerStarted","Data":"cbff2d562f3de3495e5be8a4cae37ee9cb718467898322dc391b6884f6397213"}
Jul 07 00:06:00.879230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:00.879182    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n" podStartSLOduration=1.753778166 podStartE2EDuration="2.879169602s" totalImagesPullingTime="1.125391436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:00.878660306 +0000 UTC m=+593.291674235" watchObservedRunningTime="2026-07-07 00:06:00.879169602 +0000 UTC m=+593.292183509"
Jul 07 00:06:08.039687 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:08.039657    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:06:08.047755 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:08.047724    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:06:09.026392 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:09.026338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:06:09.026592 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:09.026414    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:06:09.039130 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:09.039103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:06:09.912173 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:09.912147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n"
Jul 07 00:06:38.684141 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.684044    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"]
Jul 07 00:06:38.690857 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.690832    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.693478 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.693439    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 07 00:06:38.697845 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.697814    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"]
Jul 07 00:06:38.801142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.801109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zdq4\" (UniqueName: \"kubernetes.io/projected/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-kube-api-access-9zdq4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.801324 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.801169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.801324 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.801228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.801398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.801323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.801398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.801359    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.801501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.801448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.902931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.902893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.902931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.902929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903189 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903153    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zdq4\" (UniqueName: \"kubernetes.io/projected/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-kube-api-access-9zdq4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903404 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903374    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903484 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903413    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.903484 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.903447    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.905355 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.905331    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.905576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.905557    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:38.910746 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:38.910723    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zdq4\" (UniqueName: \"kubernetes.io/projected/6b451d6d-5945-4a7d-b4bd-a377b90fad7b-kube-api-access-9zdq4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k2fn6\" (UID: \"6b451d6d-5945-4a7d-b4bd-a377b90fad7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:39.002576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:39.002478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:39.140015 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:39.139983    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"]
Jul 07 00:06:39.143135 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:06:39.143097    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b451d6d_5945_4a7d_b4bd_a377b90fad7b.slice/crio-d4feac43843b5a2f0fe12c7a63e51fa966f0e0e476baca7705b4706c0037f151 WatchSource:0}: Error finding container d4feac43843b5a2f0fe12c7a63e51fa966f0e0e476baca7705b4706c0037f151: Status 404 returned error can't find the container with id d4feac43843b5a2f0fe12c7a63e51fa966f0e0e476baca7705b4706c0037f151
Jul 07 00:06:39.144476 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:39.144442    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:06:40.047463 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:40.047418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6" event={"ID":"6b451d6d-5945-4a7d-b4bd-a377b90fad7b","Type":"ContainerStarted","Data":"6a66161d6668562446637d3613c811cc6d9ed72f581adad261908946b0728dc5"}
Jul 07 00:06:40.047848 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:40.047550    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6" event={"ID":"6b451d6d-5945-4a7d-b4bd-a377b90fad7b","Type":"ContainerStarted","Data":"d4feac43843b5a2f0fe12c7a63e51fa966f0e0e476baca7705b4706c0037f151"}
Jul 07 00:06:40.066159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:40.066106    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6" podStartSLOduration=1.878631626 podStartE2EDuration="2.066089765s" totalImagesPullingTime="187.458139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:40.063493174 +0000 UTC m=+632.476507092" watchObservedRunningTime="2026-07-07 00:06:40.066089765 +0000 UTC m=+632.479103673"
Jul 07 00:06:42.296501 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.296440    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"]
Jul 07 00:06:42.300361 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.300337    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.303061 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.303034    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 00:06:42.311006 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.310981    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"]
Jul 07 00:06:42.436474 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.436418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.436645 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.436502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.436645 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.436610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmjj\" (UniqueName: \"kubernetes.io/projected/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-kube-api-access-6rmjj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.436730 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.436641    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.436730 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.436697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.436730 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.436724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.537575 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.537530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.537757 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.537593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.537757 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.537678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmjj\" (UniqueName: \"kubernetes.io/projected/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-kube-api-access-6rmjj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.537757 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.537704    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.537933 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.537755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.537933 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.537779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.538686 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.538658    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.538823 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.538701    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.538960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.538935    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.543479 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.541244    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.543479 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.541783    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.551358 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.551286    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmjj\" (UniqueName: \"kubernetes.io/projected/fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555-kube-api-access-6rmjj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx\" (UID: \"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.611058 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.611019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:42.748245 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:42.748219    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"]
Jul 07 00:06:42.749788 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:06:42.749753    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcf724a0_dfe2_4d5f_adb2_fd7d0ed70555.slice/crio-92cc65f0b9e60dc6a63549e47e3f5fe1669aecd6001769b815717924c5b0e27e WatchSource:0}: Error finding container 92cc65f0b9e60dc6a63549e47e3f5fe1669aecd6001769b815717924c5b0e27e: Status 404 returned error can't find the container with id 92cc65f0b9e60dc6a63549e47e3f5fe1669aecd6001769b815717924c5b0e27e
Jul 07 00:06:43.066642 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:43.066596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx" event={"ID":"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555","Type":"ContainerStarted","Data":"4b28c723a0318adb4ebe640a9453f02dd33591d92188d6d8756f75848eb01c7b"}
Jul 07 00:06:43.066642 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:43.066648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx" event={"ID":"fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555","Type":"ContainerStarted","Data":"92cc65f0b9e60dc6a63549e47e3f5fe1669aecd6001769b815717924c5b0e27e"}
Jul 07 00:06:43.085977 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:43.085918    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx" podStartSLOduration=0.889304101 podStartE2EDuration="1.085902505s" totalImagesPullingTime="196.598404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:43.08372196 +0000 UTC m=+635.496735869" watchObservedRunningTime="2026-07-07 00:06:43.085902505 +0000 UTC m=+635.498916413"
Jul 07 00:06:49.003004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:49.002950    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:49.003491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:49.003037    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:49.015561 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:49.015536    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:49.108586 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:49.108551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k2fn6"
Jul 07 00:06:52.611384 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:52.611340    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:52.611812 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:52.611405    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:52.624219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:52.624186    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:06:53.127256 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:06:53.127224    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx"
Jul 07 00:08:05.260049 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.259969    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c8db5d754-stvkt"]
Jul 07 00:08:05.263542 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.263524    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.275044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.275022    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c8db5d754-stvkt"]
Jul 07 00:08:05.394791 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.394758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afec69ca-692e-4ae8-8b06-5f7cc001bbe9-tls-cert\") pod \"authorino-c8db5d754-stvkt\" (UID: \"afec69ca-692e-4ae8-8b06-5f7cc001bbe9\") " pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.394977 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.394798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49t6k\" (UniqueName: \"kubernetes.io/projected/afec69ca-692e-4ae8-8b06-5f7cc001bbe9-kube-api-access-49t6k\") pod \"authorino-c8db5d754-stvkt\" (UID: \"afec69ca-692e-4ae8-8b06-5f7cc001bbe9\") " pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.495957 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.495924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afec69ca-692e-4ae8-8b06-5f7cc001bbe9-tls-cert\") pod \"authorino-c8db5d754-stvkt\" (UID: \"afec69ca-692e-4ae8-8b06-5f7cc001bbe9\") " pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.496108 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.495965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49t6k\" (UniqueName: \"kubernetes.io/projected/afec69ca-692e-4ae8-8b06-5f7cc001bbe9-kube-api-access-49t6k\") pod \"authorino-c8db5d754-stvkt\" (UID: \"afec69ca-692e-4ae8-8b06-5f7cc001bbe9\") " pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.498392 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.498360    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afec69ca-692e-4ae8-8b06-5f7cc001bbe9-tls-cert\") pod \"authorino-c8db5d754-stvkt\" (UID: \"afec69ca-692e-4ae8-8b06-5f7cc001bbe9\") " pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.503784 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.503762    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49t6k\" (UniqueName: \"kubernetes.io/projected/afec69ca-692e-4ae8-8b06-5f7cc001bbe9-kube-api-access-49t6k\") pod \"authorino-c8db5d754-stvkt\" (UID: \"afec69ca-692e-4ae8-8b06-5f7cc001bbe9\") " pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.572622 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.572560    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c8db5d754-stvkt"
Jul 07 00:08:05.735547 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:05.735519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c8db5d754-stvkt"]
Jul 07 00:08:05.737620 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:08:05.737588    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafec69ca_692e_4ae8_8b06_5f7cc001bbe9.slice/crio-86b0e87e0b2f140834d6d8e9a7db53e0ef508bca0297636bf21bd4b424151a8c WatchSource:0}: Error finding container 86b0e87e0b2f140834d6d8e9a7db53e0ef508bca0297636bf21bd4b424151a8c: Status 404 returned error can't find the container with id 86b0e87e0b2f140834d6d8e9a7db53e0ef508bca0297636bf21bd4b424151a8c
Jul 07 00:08:06.474247 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.474210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c8db5d754-stvkt" event={"ID":"afec69ca-692e-4ae8-8b06-5f7cc001bbe9","Type":"ContainerStarted","Data":"077f35f025e3c9efa945823ce5e2d59fcf06273b01b15f6c50e4b15bc8abd936"}
Jul 07 00:08:06.474715 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.474269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c8db5d754-stvkt" event={"ID":"afec69ca-692e-4ae8-8b06-5f7cc001bbe9","Type":"ContainerStarted","Data":"86b0e87e0b2f140834d6d8e9a7db53e0ef508bca0297636bf21bd4b424151a8c"}
Jul 07 00:08:06.492140 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.491971    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c8db5d754-stvkt" podStartSLOduration=0.968494113 podStartE2EDuration="1.491950355s" totalImagesPullingTime="523.456242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:06.491710884 +0000 UTC m=+718.904724794" watchObservedRunningTime="2026-07-07 00:08:06.491950355 +0000 UTC m=+718.904964264"
Jul 07 00:08:06.520410 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.519043    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54cffd5dc9-5gwd8"]
Jul 07 00:08:06.520410 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.519285    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8" podUID="f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" containerName="authorino" containerID="cri-o://3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a" gracePeriod=30
Jul 07 00:08:06.785921 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.785823    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:08:06.909304 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.909268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfpg6\" (UniqueName: \"kubernetes.io/projected/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-kube-api-access-gfpg6\") pod \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") "
Jul 07 00:08:06.909304 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.909306    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-tls-cert\") pod \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\" (UID: \"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f\") "
Jul 07 00:08:06.911425 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.911395    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-kube-api-access-gfpg6" (OuterVolumeSpecName: "kube-api-access-gfpg6") pod "f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" (UID: "f6bf9f27-83e4-4f71-9749-d8f9500c3f6f"). InnerVolumeSpecName "kube-api-access-gfpg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:08:06.920900 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:06.920869    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" (UID: "f6bf9f27-83e4-4f71-9749-d8f9500c3f6f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:08:07.010564 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.010528    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfpg6\" (UniqueName: \"kubernetes.io/projected/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-kube-api-access-gfpg6\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:08:07.010564 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.010561    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f-tls-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:08:07.481638 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.481604    2564 generic.go:356] "Generic (PLEG): container finished" podID="f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" containerID="3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a" exitCode=0
Jul 07 00:08:07.482120 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.481650    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8"
Jul 07 00:08:07.482120 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.481686    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8" event={"ID":"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f","Type":"ContainerDied","Data":"3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a"}
Jul 07 00:08:07.482120 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.481740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54cffd5dc9-5gwd8" event={"ID":"f6bf9f27-83e4-4f71-9749-d8f9500c3f6f","Type":"ContainerDied","Data":"f839f185582ae120060eac8d74b6753cf41d6378c5742a63a7c4f9bab172b748"}
Jul 07 00:08:07.482120 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.481771    2564 scope.go:117] "RemoveContainer" containerID="3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a"
Jul 07 00:08:07.492646 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.492620    2564 scope.go:117] "RemoveContainer" containerID="3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a"
Jul 07 00:08:07.492895 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:08:07.492877    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a\": container with ID starting with 3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a not found: ID does not exist" containerID="3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a"
Jul 07 00:08:07.492954 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.492910    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a"} err="failed to get container status \"3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a\": rpc error: code = NotFound desc = could not find container \"3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a\": container with ID starting with 3156e219744f0083a4d75d091ead488af198bb235b7b8d9a9a0626748523712a not found: ID does not exist"
Jul 07 00:08:07.503864 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.503834    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54cffd5dc9-5gwd8"]
Jul 07 00:08:07.510059 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:07.510031    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54cffd5dc9-5gwd8"]
Jul 07 00:08:08.115293 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:08:08.115261    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" path="/var/lib/kubelet/pods/f6bf9f27-83e4-4f71-9749-d8f9500c3f6f/volumes"
Jul 07 00:09:27.314566 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.314526    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6966b9487-nr5cj"]
Jul 07 00:09:27.315493 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.314791    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6966b9487-nr5cj" podUID="5c43d94e-50e3-4c79-8b3e-3a538169c15a" containerName="manager" containerID="cri-o://8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6" gracePeriod=30
Jul 07 00:09:27.558655 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.558634    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:09:27.654831 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.654798    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z5qm\" (UniqueName: \"kubernetes.io/projected/5c43d94e-50e3-4c79-8b3e-3a538169c15a-kube-api-access-5z5qm\") pod \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") "
Jul 07 00:09:27.655002 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.654836    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c43d94e-50e3-4c79-8b3e-3a538169c15a-webhook-cert\") pod \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\" (UID: \"5c43d94e-50e3-4c79-8b3e-3a538169c15a\") "
Jul 07 00:09:27.656794 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.656768    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c43d94e-50e3-4c79-8b3e-3a538169c15a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5c43d94e-50e3-4c79-8b3e-3a538169c15a" (UID: "5c43d94e-50e3-4c79-8b3e-3a538169c15a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:09:27.656794 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.656781    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c43d94e-50e3-4c79-8b3e-3a538169c15a-kube-api-access-5z5qm" (OuterVolumeSpecName: "kube-api-access-5z5qm") pod "5c43d94e-50e3-4c79-8b3e-3a538169c15a" (UID: "5c43d94e-50e3-4c79-8b3e-3a538169c15a"). InnerVolumeSpecName "kube-api-access-5z5qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:09:27.755661 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.755634    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z5qm\" (UniqueName: \"kubernetes.io/projected/5c43d94e-50e3-4c79-8b3e-3a538169c15a-kube-api-access-5z5qm\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:09:27.755661 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.755657    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c43d94e-50e3-4c79-8b3e-3a538169c15a-webhook-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:09:27.858051 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.858025    2564 generic.go:356] "Generic (PLEG): container finished" podID="5c43d94e-50e3-4c79-8b3e-3a538169c15a" containerID="8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6" exitCode=0
Jul 07 00:09:27.858200 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.858081    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6966b9487-nr5cj"
Jul 07 00:09:27.858200 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.858113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6966b9487-nr5cj" event={"ID":"5c43d94e-50e3-4c79-8b3e-3a538169c15a","Type":"ContainerDied","Data":"8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6"}
Jul 07 00:09:27.858200 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.858168    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6966b9487-nr5cj" event={"ID":"5c43d94e-50e3-4c79-8b3e-3a538169c15a","Type":"ContainerDied","Data":"94be91d7061206de133ff32868ebe0d3d76f3dbc7a5e4d9d7f482895e9ad4679"}
Jul 07 00:09:27.858200 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.858198    2564 scope.go:117] "RemoveContainer" containerID="8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6"
Jul 07 00:09:27.867278 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.867260    2564 scope.go:117] "RemoveContainer" containerID="8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6"
Jul 07 00:09:27.867607 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:09:27.867588    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6\": container with ID starting with 8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6 not found: ID does not exist" containerID="8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6"
Jul 07 00:09:27.867666 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.867616    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6"} err="failed to get container status \"8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6\": rpc error: code = NotFound desc = could not find container \"8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6\": container with ID starting with 8d1a3684e0bed869b05b9bb89a56e74ab6924d042f066fbc53c662ef35d999a6 not found: ID does not exist"
Jul 07 00:09:27.878332 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.878312    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6966b9487-nr5cj"]
Jul 07 00:09:27.881818 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:27.881798    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6966b9487-nr5cj"]
Jul 07 00:09:28.116216 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.116188    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c43d94e-50e3-4c79-8b3e-3a538169c15a" path="/var/lib/kubelet/pods/5c43d94e-50e3-4c79-8b3e-3a538169c15a/volumes"
Jul 07 00:09:28.749008 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.748976    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6966b9487-qhhx6"]
Jul 07 00:09:28.749366 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.749341    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c43d94e-50e3-4c79-8b3e-3a538169c15a" containerName="manager"
Jul 07 00:09:28.749366 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.749353    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c43d94e-50e3-4c79-8b3e-3a538169c15a" containerName="manager"
Jul 07 00:09:28.749438 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.749369    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" containerName="authorino"
Jul 07 00:09:28.749438 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.749375    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" containerName="authorino"
Jul 07 00:09:28.749530 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.749441    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bf9f27-83e4-4f71-9749-d8f9500c3f6f" containerName="authorino"
Jul 07 00:09:28.749530 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.749464    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c43d94e-50e3-4c79-8b3e-3a538169c15a" containerName="manager"
Jul 07 00:09:28.753797 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.753777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:28.757681 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.757655    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 00:09:28.757788 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.757729    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-98rhj\""
Jul 07 00:09:28.757846 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.757788    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 00:09:28.762994 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.762972    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6966b9487-qhhx6"]
Jul 07 00:09:28.863808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.863781    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp5zx\" (UniqueName: \"kubernetes.io/projected/b4ecc55e-30a9-4b5f-b853-e29cb7ef6785-kube-api-access-jp5zx\") pod \"maas-controller-6966b9487-qhhx6\" (UID: \"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785\") " pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:28.863985 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.863827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4ecc55e-30a9-4b5f-b853-e29cb7ef6785-webhook-cert\") pod \"maas-controller-6966b9487-qhhx6\" (UID: \"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785\") " pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:28.964882 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.964854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp5zx\" (UniqueName: \"kubernetes.io/projected/b4ecc55e-30a9-4b5f-b853-e29cb7ef6785-kube-api-access-jp5zx\") pod \"maas-controller-6966b9487-qhhx6\" (UID: \"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785\") " pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:28.965010 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.964895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4ecc55e-30a9-4b5f-b853-e29cb7ef6785-webhook-cert\") pod \"maas-controller-6966b9487-qhhx6\" (UID: \"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785\") " pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:28.967375 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.967346    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4ecc55e-30a9-4b5f-b853-e29cb7ef6785-webhook-cert\") pod \"maas-controller-6966b9487-qhhx6\" (UID: \"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785\") " pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:28.973151 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:28.973122    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp5zx\" (UniqueName: \"kubernetes.io/projected/b4ecc55e-30a9-4b5f-b853-e29cb7ef6785-kube-api-access-jp5zx\") pod \"maas-controller-6966b9487-qhhx6\" (UID: \"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785\") " pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:29.064142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:29.064082    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:29.201163 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:29.201139    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6966b9487-qhhx6"]
Jul 07 00:09:29.203191 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:09:29.203164    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ecc55e_30a9_4b5f_b853_e29cb7ef6785.slice/crio-19e6139ec21f6bffe49f636f6d7d7b8e53f9e4af427beafbf227f648a820c068 WatchSource:0}: Error finding container 19e6139ec21f6bffe49f636f6d7d7b8e53f9e4af427beafbf227f648a820c068: Status 404 returned error can't find the container with id 19e6139ec21f6bffe49f636f6d7d7b8e53f9e4af427beafbf227f648a820c068
Jul 07 00:09:29.870920 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:29.870883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6966b9487-qhhx6" event={"ID":"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785","Type":"ContainerStarted","Data":"816b99542dae8c9554df01dec8965f9abbeddd9166c519b0e3629dbffb72d13b"}
Jul 07 00:09:29.871364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:29.870953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6966b9487-qhhx6" event={"ID":"b4ecc55e-30a9-4b5f-b853-e29cb7ef6785","Type":"ContainerStarted","Data":"19e6139ec21f6bffe49f636f6d7d7b8e53f9e4af427beafbf227f648a820c068"}
Jul 07 00:09:29.871364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:29.870983    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:09:29.887422 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:29.887381    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6966b9487-qhhx6" podStartSLOduration=1.348576069 podStartE2EDuration="1.887363943s" totalImagesPullingTime="538.787874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:09:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:09:29.886734064 +0000 UTC m=+802.299747971" watchObservedRunningTime="2026-07-07 00:09:29.887363943 +0000 UTC m=+802.300377865"
Jul 07 00:09:40.881807 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:09:40.881726    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6966b9487-qhhx6"
Jul 07 00:11:08.070917 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:11:08.070840    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:11:08.076158 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:11:08.076137    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:16:08.103161 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:16:08.103133    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:16:08.105693 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:16:08.105502    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:21:08.144504 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:08.144379    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:21:08.144504 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:08.144419    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:21:32.291591 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.291561    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"]
Jul 07 00:21:32.294520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.294499    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.297555 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.297534    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-30953ec9-openshift-default-dockercfg-pqs6z\""
Jul 07 00:21:32.321989 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.321949    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"]
Jul 07 00:21:32.352025 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.351993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjdn\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-kube-api-access-4qjdn\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-envoy\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-podinfo\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-token\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-socket\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-credential-socket\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352406 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-data\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352406 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-certs\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.352406 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.352386    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6790e006-800e-4a46-8d18-9ef8bb3333ff-istiod-ca-cert\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.453895 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.453854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjdn\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-kube-api-access-4qjdn\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454106 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.453917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-envoy\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454106 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.453943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-podinfo\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454106 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.453993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-token\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-socket\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-credential-socket\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454226    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-data\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-certs\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454562 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6790e006-800e-4a46-8d18-9ef8bb3333ff-istiod-ca-cert\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454562 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454500    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-socket\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454652 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454580    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-credential-socket\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.454652 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454634    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-data\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.455110 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454688    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-certs\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.455110 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.454980    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6790e006-800e-4a46-8d18-9ef8bb3333ff-istiod-ca-cert\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.456320 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.456298    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-envoy\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.456543 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.456524    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-podinfo\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.462749 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.462725    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-token\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.462902 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.462884    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjdn\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-kube-api-access-4qjdn\") pod \"e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") " pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.605125 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.605088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:32.767506 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.767474    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"]
Jul 07 00:21:32.773274 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:21:32.773245    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6790e006_800e_4a46_8d18_9ef8bb3333ff.slice/crio-4e10584dad1c1725110b2c2f56cc661c2e6437edf915f5faae5ae291a65782e3 WatchSource:0}: Error finding container 4e10584dad1c1725110b2c2f56cc661c2e6437edf915f5faae5ae291a65782e3: Status 404 returned error can't find the container with id 4e10584dad1c1725110b2c2f56cc661c2e6437edf915f5faae5ae291a65782e3
Jul 07 00:21:32.775443 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.775408    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:21:32.775556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.775492    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:21:32.775556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:32.775523    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:21:33.392436 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.392393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb" event={"ID":"6790e006-800e-4a46-8d18-9ef8bb3333ff","Type":"ContainerStarted","Data":"04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e"}
Jul 07 00:21:33.392872 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.392487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb" event={"ID":"6790e006-800e-4a46-8d18-9ef8bb3333ff","Type":"ContainerStarted","Data":"4e10584dad1c1725110b2c2f56cc661c2e6437edf915f5faae5ae291a65782e3"}
Jul 07 00:21:33.424279 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.424227    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb" podStartSLOduration=1.424207238 podStartE2EDuration="1.424207238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:33.423092695 +0000 UTC m=+1525.836106605" watchObservedRunningTime="2026-07-07 00:21:33.424207238 +0000 UTC m=+1525.837221147"
Jul 07 00:21:33.518556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.518513    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"]
Jul 07 00:21:33.522354 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.522329    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.525125 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.525105    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-30953ec9-serving-cert\""
Jul 07 00:21:33.531593 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.531567    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"]
Jul 07 00:21:33.605758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.605730    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:33.610291 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.610268    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:33.666248 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.666163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5666b653-9ea4-4c63-8109-979fae0ccb59-openshift-service-ca\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.666248 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.666221    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5666b653-9ea4-4c63-8109-979fae0ccb59-maas-api-tls\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.666479 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.666286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc9s4\" (UniqueName: \"kubernetes.io/projected/5666b653-9ea4-4c63-8109-979fae0ccb59-kube-api-access-pc9s4\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.767212 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.767168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5666b653-9ea4-4c63-8109-979fae0ccb59-openshift-service-ca\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.767428 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.767339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5666b653-9ea4-4c63-8109-979fae0ccb59-maas-api-tls\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.767428 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.767378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc9s4\" (UniqueName: \"kubernetes.io/projected/5666b653-9ea4-4c63-8109-979fae0ccb59-kube-api-access-pc9s4\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.767927 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.767903    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5666b653-9ea4-4c63-8109-979fae0ccb59-openshift-service-ca\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.769784 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.769763    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5666b653-9ea4-4c63-8109-979fae0ccb59-maas-api-tls\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.775127 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.775107    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc9s4\" (UniqueName: \"kubernetes.io/projected/5666b653-9ea4-4c63-8109-979fae0ccb59-kube-api-access-pc9s4\") pod \"maas-api-e2e-ait-30953ec9-86567bc494-2nhc2\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") " pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.835399 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.835358    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:33.979645 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.979618    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"]
Jul 07 00:21:33.981446 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:21:33.981421    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5666b653_9ea4_4c63_8109_979fae0ccb59.slice/crio-e23baa77cac4a7489a30d002a3ab0f9c2eabacb1df714d81193d9abe38af1d00 WatchSource:0}: Error finding container e23baa77cac4a7489a30d002a3ab0f9c2eabacb1df714d81193d9abe38af1d00: Status 404 returned error can't find the container with id e23baa77cac4a7489a30d002a3ab0f9c2eabacb1df714d81193d9abe38af1d00
Jul 07 00:21:33.982950 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:33.982932    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:21:34.399131 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:34.399083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2" event={"ID":"5666b653-9ea4-4c63-8109-979fae0ccb59","Type":"ContainerStarted","Data":"e23baa77cac4a7489a30d002a3ab0f9c2eabacb1df714d81193d9abe38af1d00"}
Jul 07 00:21:34.399576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:34.399390    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:34.400225 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:34.400205    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:35.408425 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:35.408390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2" event={"ID":"5666b653-9ea4-4c63-8109-979fae0ccb59","Type":"ContainerStarted","Data":"f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5"}
Jul 07 00:21:35.408865 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:35.408586    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:35.425084 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:35.425035    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2" podStartSLOduration=1.818788553 podStartE2EDuration="2.425018306s" totalImagesPullingTime="606.229753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:35.423736489 +0000 UTC m=+1527.836750400" watchObservedRunningTime="2026-07-07 00:21:35.425018306 +0000 UTC m=+1527.838032214"
Jul 07 00:21:38.755691 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:38.755616    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"]
Jul 07 00:21:38.756391 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:38.755916    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb" podUID="6790e006-800e-4a46-8d18-9ef8bb3333ff" containerName="istio-proxy" containerID="cri-o://04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e" gracePeriod=30
Jul 07 00:21:38.951205 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:38.951167    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"]
Jul 07 00:21:38.951724 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:38.951669    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2" podUID="5666b653-9ea4-4c63-8109-979fae0ccb59" containerName="maas-api" containerID="cri-o://f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5" gracePeriod=30
Jul 07 00:21:39.265229 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.265205    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:39.421875 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.421844    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5666b653-9ea4-4c63-8109-979fae0ccb59-openshift-service-ca\") pod \"5666b653-9ea4-4c63-8109-979fae0ccb59\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") "
Jul 07 00:21:39.422096 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.421929    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc9s4\" (UniqueName: \"kubernetes.io/projected/5666b653-9ea4-4c63-8109-979fae0ccb59-kube-api-access-pc9s4\") pod \"5666b653-9ea4-4c63-8109-979fae0ccb59\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") "
Jul 07 00:21:39.422096 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.421953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5666b653-9ea4-4c63-8109-979fae0ccb59-maas-api-tls\") pod \"5666b653-9ea4-4c63-8109-979fae0ccb59\" (UID: \"5666b653-9ea4-4c63-8109-979fae0ccb59\") "
Jul 07 00:21:39.422230 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.422192    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5666b653-9ea4-4c63-8109-979fae0ccb59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5666b653-9ea4-4c63-8109-979fae0ccb59" (UID: "5666b653-9ea4-4c63-8109-979fae0ccb59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:21:39.424113 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.424068    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5666b653-9ea4-4c63-8109-979fae0ccb59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5666b653-9ea4-4c63-8109-979fae0ccb59" (UID: "5666b653-9ea4-4c63-8109-979fae0ccb59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:21:39.424113 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.424068    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5666b653-9ea4-4c63-8109-979fae0ccb59-kube-api-access-pc9s4" (OuterVolumeSpecName: "kube-api-access-pc9s4") pod "5666b653-9ea4-4c63-8109-979fae0ccb59" (UID: "5666b653-9ea4-4c63-8109-979fae0ccb59"). InnerVolumeSpecName "kube-api-access-pc9s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:21:39.434477 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.430750    2564 generic.go:356] "Generic (PLEG): container finished" podID="5666b653-9ea4-4c63-8109-979fae0ccb59" containerID="f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5" exitCode=0
Jul 07 00:21:39.434477 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.430879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2" event={"ID":"5666b653-9ea4-4c63-8109-979fae0ccb59","Type":"ContainerDied","Data":"f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5"}
Jul 07 00:21:39.434477 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.430928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2" event={"ID":"5666b653-9ea4-4c63-8109-979fae0ccb59","Type":"ContainerDied","Data":"e23baa77cac4a7489a30d002a3ab0f9c2eabacb1df714d81193d9abe38af1d00"}
Jul 07 00:21:39.434477 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.430959    2564 scope.go:117] "RemoveContainer" containerID="f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5"
Jul 07 00:21:39.434477 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.431125    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"
Jul 07 00:21:39.463122 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.463100    2564 scope.go:117] "RemoveContainer" containerID="f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5"
Jul 07 00:21:39.463563 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:21:39.463530    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5\": container with ID starting with f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5 not found: ID does not exist" containerID="f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5"
Jul 07 00:21:39.463698 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.463591    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5"} err="failed to get container status \"f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5\": rpc error: code = NotFound desc = could not find container \"f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5\": container with ID starting with f759cacb199f38a379d948016512e2d75dfd97d85ba4369a2c7cf9bfd45355c5 not found: ID does not exist"
Jul 07 00:21:39.473222 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.473195    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"]
Jul 07 00:21:39.477524 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.477501    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-30953ec9-86567bc494-2nhc2"]
Jul 07 00:21:39.523368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.523335    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5666b653-9ea4-4c63-8109-979fae0ccb59-openshift-service-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:39.523368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.523362    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc9s4\" (UniqueName: \"kubernetes.io/projected/5666b653-9ea4-4c63-8109-979fae0ccb59-kube-api-access-pc9s4\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:39.523368 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:39.523375    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5666b653-9ea4-4c63-8109-979fae0ccb59-maas-api-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:40.114336 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:40.114309    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5666b653-9ea4-4c63-8109-979fae0ccb59" path="/var/lib/kubelet/pods/5666b653-9ea4-4c63-8109-979fae0ccb59/volumes"
Jul 07 00:21:44.617247 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.617222    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:44.780398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780308    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6790e006-800e-4a46-8d18-9ef8bb3333ff-istiod-ca-cert\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780369    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-envoy\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780391    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-credential-socket\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780709 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780435    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qjdn\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-kube-api-access-4qjdn\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780709 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780503    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-socket\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780709 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780541    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-podinfo\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780709 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-token\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780709 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780611    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-certs\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780709 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-data\") pod \"6790e006-800e-4a46-8d18-9ef8bb3333ff\" (UID: \"6790e006-800e-4a46-8d18-9ef8bb3333ff\") "
Jul 07 00:21:44.780996 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780853    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:21:44.780996 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.780973    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.781108 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.781089    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:21:44.781200 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.781121    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:21:44.781347 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.781192    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6790e006-800e-4a46-8d18-9ef8bb3333ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:21:44.781759 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.781734    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:21:44.782835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.782808    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:21:44.782945 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.782879    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:21:44.783001 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.782953    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:21:44.783277 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.783255    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-kube-api-access-4qjdn" (OuterVolumeSpecName: "kube-api-access-4qjdn") pod "6790e006-800e-4a46-8d18-9ef8bb3333ff" (UID: "6790e006-800e-4a46-8d18-9ef8bb3333ff"). InnerVolumeSpecName "kube-api-access-4qjdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:21:44.881695 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881668    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-podinfo\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881695 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881692    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881695 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881701    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-workload-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881911 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881709    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-data\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881911 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881717    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6790e006-800e-4a46-8d18-9ef8bb3333ff-istiod-ca-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881911 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881724    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-istio-envoy\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881911 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881731    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6790e006-800e-4a46-8d18-9ef8bb3333ff-credential-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:44.881911 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:44.881741    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qjdn\" (UniqueName: \"kubernetes.io/projected/6790e006-800e-4a46-8d18-9ef8bb3333ff-kube-api-access-4qjdn\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:45.470447 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.470355    2564 generic.go:356] "Generic (PLEG): container finished" podID="6790e006-800e-4a46-8d18-9ef8bb3333ff" containerID="04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e" exitCode=0
Jul 07 00:21:45.470447 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.470399    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"
Jul 07 00:21:45.470447 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.470434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb" event={"ID":"6790e006-800e-4a46-8d18-9ef8bb3333ff","Type":"ContainerDied","Data":"04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e"}
Jul 07 00:21:45.470771 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.470501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb" event={"ID":"6790e006-800e-4a46-8d18-9ef8bb3333ff","Type":"ContainerDied","Data":"4e10584dad1c1725110b2c2f56cc661c2e6437edf915f5faae5ae291a65782e3"}
Jul 07 00:21:45.470771 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.470534    2564 scope.go:117] "RemoveContainer" containerID="04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e"
Jul 07 00:21:45.480345 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.480160    2564 scope.go:117] "RemoveContainer" containerID="04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e"
Jul 07 00:21:45.480482 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:21:45.480441    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e\": container with ID starting with 04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e not found: ID does not exist" containerID="04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e"
Jul 07 00:21:45.480585 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.480490    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e"} err="failed to get container status \"04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e\": rpc error: code = NotFound desc = could not find container \"04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e\": container with ID starting with 04daea0927551b19ac7a92cf0198b50851745e30515e0e6caea6cb3a1b69220e not found: ID does not exist"
Jul 07 00:21:45.497787 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.497760    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"]
Jul 07 00:21:45.500445 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:45.500424    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-30953ec9-openshift-default-59fcc7b85c-l2xwb"]
Jul 07 00:21:46.114468 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.114426    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6790e006-800e-4a46-8d18-9ef8bb3333ff" path="/var/lib/kubelet/pods/6790e006-800e-4a46-8d18-9ef8bb3333ff/volumes"
Jul 07 00:21:46.401985 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.401910    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"]
Jul 07 00:21:46.402324 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.402313    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6790e006-800e-4a46-8d18-9ef8bb3333ff" containerName="istio-proxy"
Jul 07 00:21:46.402374 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.402326    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6790e006-800e-4a46-8d18-9ef8bb3333ff" containerName="istio-proxy"
Jul 07 00:21:46.402374 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.402339    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5666b653-9ea4-4c63-8109-979fae0ccb59" containerName="maas-api"
Jul 07 00:21:46.402374 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.402346    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5666b653-9ea4-4c63-8109-979fae0ccb59" containerName="maas-api"
Jul 07 00:21:46.402486 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.402403    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6790e006-800e-4a46-8d18-9ef8bb3333ff" containerName="istio-proxy"
Jul 07 00:21:46.402486 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.402416    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5666b653-9ea4-4c63-8109-979fae0ccb59" containerName="maas-api"
Jul 07 00:21:46.406849 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.406831    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.412124 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.412103    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-7a16f626-serving-cert\""
Jul 07 00:21:46.445260 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.445232    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"]
Jul 07 00:21:46.496135 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.496104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5110494f-0ebe-4cb2-96d2-92076bd9c378-openshift-service-ca\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.496314 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.496149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5110494f-0ebe-4cb2-96d2-92076bd9c378-maas-api-tls\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.496314 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.496212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6bk2\" (UniqueName: \"kubernetes.io/projected/5110494f-0ebe-4cb2-96d2-92076bd9c378-kube-api-access-h6bk2\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.597848 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.597808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5110494f-0ebe-4cb2-96d2-92076bd9c378-openshift-service-ca\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.598018 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.597860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5110494f-0ebe-4cb2-96d2-92076bd9c378-maas-api-tls\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.598018 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.597905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6bk2\" (UniqueName: \"kubernetes.io/projected/5110494f-0ebe-4cb2-96d2-92076bd9c378-kube-api-access-h6bk2\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.598427 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.598407    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5110494f-0ebe-4cb2-96d2-92076bd9c378-openshift-service-ca\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.600349 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.600327    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5110494f-0ebe-4cb2-96d2-92076bd9c378-maas-api-tls\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.622140 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.622117    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6bk2\" (UniqueName: \"kubernetes.io/projected/5110494f-0ebe-4cb2-96d2-92076bd9c378-kube-api-access-h6bk2\") pod \"maas-api-e2e-ait-7a16f626-7944746896-22hnd\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") " pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.716721 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.716638    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:46.859720 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:46.859696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"]
Jul 07 00:21:46.861372 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:21:46.861347    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5110494f_0ebe_4cb2_96d2_92076bd9c378.slice/crio-64faafaebc581b7831810a32282b10f78b4cd43504f4de1f33e8c5ef2c4be3b0 WatchSource:0}: Error finding container 64faafaebc581b7831810a32282b10f78b4cd43504f4de1f33e8c5ef2c4be3b0: Status 404 returned error can't find the container with id 64faafaebc581b7831810a32282b10f78b4cd43504f4de1f33e8c5ef2c4be3b0
Jul 07 00:21:47.489084 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:47.489054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd" event={"ID":"5110494f-0ebe-4cb2-96d2-92076bd9c378","Type":"ContainerStarted","Data":"e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde"}
Jul 07 00:21:47.489399 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:47.489105    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd" event={"ID":"5110494f-0ebe-4cb2-96d2-92076bd9c378","Type":"ContainerStarted","Data":"64faafaebc581b7831810a32282b10f78b4cd43504f4de1f33e8c5ef2c4be3b0"}
Jul 07 00:21:47.489399 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:47.489186    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:47.511801 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:47.511758    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd" podStartSLOduration=0.971047386 podStartE2EDuration="1.511743869s" totalImagesPullingTime="540.696483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:47.510035085 +0000 UTC m=+1539.923048992" watchObservedRunningTime="2026-07-07 00:21:47.511743869 +0000 UTC m=+1539.924757777"
Jul 07 00:21:51.706736 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:51.706703    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"]
Jul 07 00:21:51.707142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:51.706898    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd" podUID="5110494f-0ebe-4cb2-96d2-92076bd9c378" containerName="maas-api" containerID="cri-o://e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde" gracePeriod=30
Jul 07 00:21:51.953741 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:51.953719    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:52.149429 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.149403    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5110494f-0ebe-4cb2-96d2-92076bd9c378-maas-api-tls\") pod \"5110494f-0ebe-4cb2-96d2-92076bd9c378\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") "
Jul 07 00:21:52.149634 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.149478    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5110494f-0ebe-4cb2-96d2-92076bd9c378-openshift-service-ca\") pod \"5110494f-0ebe-4cb2-96d2-92076bd9c378\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") "
Jul 07 00:21:52.149634 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.149561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6bk2\" (UniqueName: \"kubernetes.io/projected/5110494f-0ebe-4cb2-96d2-92076bd9c378-kube-api-access-h6bk2\") pod \"5110494f-0ebe-4cb2-96d2-92076bd9c378\" (UID: \"5110494f-0ebe-4cb2-96d2-92076bd9c378\") "
Jul 07 00:21:52.149855 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.149832    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5110494f-0ebe-4cb2-96d2-92076bd9c378-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5110494f-0ebe-4cb2-96d2-92076bd9c378" (UID: "5110494f-0ebe-4cb2-96d2-92076bd9c378"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:21:52.151520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.151489    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5110494f-0ebe-4cb2-96d2-92076bd9c378-kube-api-access-h6bk2" (OuterVolumeSpecName: "kube-api-access-h6bk2") pod "5110494f-0ebe-4cb2-96d2-92076bd9c378" (UID: "5110494f-0ebe-4cb2-96d2-92076bd9c378"). InnerVolumeSpecName "kube-api-access-h6bk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:21:52.151629 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.151517    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5110494f-0ebe-4cb2-96d2-92076bd9c378-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5110494f-0ebe-4cb2-96d2-92076bd9c378" (UID: "5110494f-0ebe-4cb2-96d2-92076bd9c378"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:21:52.251142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.251110    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6bk2\" (UniqueName: \"kubernetes.io/projected/5110494f-0ebe-4cb2-96d2-92076bd9c378-kube-api-access-h6bk2\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:52.251142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.251140    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5110494f-0ebe-4cb2-96d2-92076bd9c378-maas-api-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:52.251317 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.251152    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5110494f-0ebe-4cb2-96d2-92076bd9c378-openshift-service-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:21:52.517113 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.517011    2564 generic.go:356] "Generic (PLEG): container finished" podID="5110494f-0ebe-4cb2-96d2-92076bd9c378" containerID="e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde" exitCode=0
Jul 07 00:21:52.517113 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.517056    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"
Jul 07 00:21:52.517113 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.517097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd" event={"ID":"5110494f-0ebe-4cb2-96d2-92076bd9c378","Type":"ContainerDied","Data":"e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde"}
Jul 07 00:21:52.517394 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.517151    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd" event={"ID":"5110494f-0ebe-4cb2-96d2-92076bd9c378","Type":"ContainerDied","Data":"64faafaebc581b7831810a32282b10f78b4cd43504f4de1f33e8c5ef2c4be3b0"}
Jul 07 00:21:52.517394 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.517175    2564 scope.go:117] "RemoveContainer" containerID="e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde"
Jul 07 00:21:52.526210 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.526188    2564 scope.go:117] "RemoveContainer" containerID="e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde"
Jul 07 00:21:52.526445 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:21:52.526426    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde\": container with ID starting with e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde not found: ID does not exist" containerID="e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde"
Jul 07 00:21:52.526541 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.526471    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde"} err="failed to get container status \"e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde\": rpc error: code = NotFound desc = could not find container \"e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde\": container with ID starting with e2ae9832391204a2b12089a767e46de0ab67120c0d14e3e59ad1edf308838fde not found: ID does not exist"
Jul 07 00:21:52.541045 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.541021    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"]
Jul 07 00:21:52.550229 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:52.546228    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-7a16f626-7944746896-22hnd"]
Jul 07 00:21:54.114981 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:21:54.114952    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5110494f-0ebe-4cb2-96d2-92076bd9c378" path="/var/lib/kubelet/pods/5110494f-0ebe-4cb2-96d2-92076bd9c378/volumes"
Jul 07 00:22:03.767408 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.767372    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"]
Jul 07 00:22:03.768117 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.768090    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5110494f-0ebe-4cb2-96d2-92076bd9c378" containerName="maas-api"
Jul 07 00:22:03.768159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.768117    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5110494f-0ebe-4cb2-96d2-92076bd9c378" containerName="maas-api"
Jul 07 00:22:03.768247 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.768234    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5110494f-0ebe-4cb2-96d2-92076bd9c378" containerName="maas-api"
Jul 07 00:22:03.773024 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.773004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.776551 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.776531    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-70108423-openshift-default-dockercfg-mbz4p\""
Jul 07 00:22:03.788243 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.788219    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"]
Jul 07 00:22:03.859397 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-envoy\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-certs\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-token\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istiod-ca-cert\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-socket\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9pr\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-kube-api-access-9k9pr\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859891 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859674    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-credential-socket\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859891 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-podinfo\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.859891 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.859804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-data\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.960960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.960924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-envoy\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961135 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.960973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-certs\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961135 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-token\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istiod-ca-cert\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-socket\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961366 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9pr\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-kube-api-access-9k9pr\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961366 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961358    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-certs\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961600 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-credential-socket\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961600 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961510    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-socket\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961600 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-podinfo\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961600 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-data\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961829 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961708    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-credential-socket\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.961891 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.961832    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-data\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.962027 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.962003    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istiod-ca-cert\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.963438 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.963416    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-envoy\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.963852 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.963836    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-podinfo\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.969572 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.969546    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-token\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:03.969699 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:03.969682    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9pr\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-kube-api-access-9k9pr\") pod \"e2e-derive-70108423-openshift-default-75798568d7-5vsws\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") " pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:04.084575 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.084478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:04.227819 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.227794    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"]
Jul 07 00:22:04.229485 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:22:04.229438    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f38900b_b6f4_4ad7_993e_379e1d22cc33.slice/crio-fdf6c7570ef85ba1cf65ded95fc68385a10c53e2ed1be257015378a392965758 WatchSource:0}: Error finding container fdf6c7570ef85ba1cf65ded95fc68385a10c53e2ed1be257015378a392965758: Status 404 returned error can't find the container with id fdf6c7570ef85ba1cf65ded95fc68385a10c53e2ed1be257015378a392965758
Jul 07 00:22:04.232184 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.232151    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:22:04.232299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.232237    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:22:04.232299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.232279    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:22:04.588005 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.587971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws" event={"ID":"3f38900b-b6f4-4ad7-993e-379e1d22cc33","Type":"ContainerStarted","Data":"42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b"}
Jul 07 00:22:04.588235 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.588022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws" event={"ID":"3f38900b-b6f4-4ad7-993e-379e1d22cc33","Type":"ContainerStarted","Data":"fdf6c7570ef85ba1cf65ded95fc68385a10c53e2ed1be257015378a392965758"}
Jul 07 00:22:04.612329 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:04.612281    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws" podStartSLOduration=1.612264433 podStartE2EDuration="1.612264433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:22:04.61140627 +0000 UTC m=+1557.024420174" watchObservedRunningTime="2026-07-07 00:22:04.612264433 +0000 UTC m=+1557.025278341"
Jul 07 00:22:05.055508 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.055475    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"]
Jul 07 00:22:05.059498 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.059476    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.062667 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.062649    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-70108423-serving-cert\""
Jul 07 00:22:05.072644 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.072619    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"]
Jul 07 00:22:05.085662 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.085635    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:05.091211 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.091186    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:05.174782 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.174751    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d97d8c7-e547-4c48-be7f-9193caa535a3-maas-api-tls\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.174782 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.174786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgcb4\" (UniqueName: \"kubernetes.io/projected/8d97d8c7-e547-4c48-be7f-9193caa535a3-kube-api-access-vgcb4\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.175037 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.174936    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d97d8c7-e547-4c48-be7f-9193caa535a3-openshift-service-ca\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.276139 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.276107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d97d8c7-e547-4c48-be7f-9193caa535a3-openshift-service-ca\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.276327 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.276186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d97d8c7-e547-4c48-be7f-9193caa535a3-maas-api-tls\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.276327 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.276215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgcb4\" (UniqueName: \"kubernetes.io/projected/8d97d8c7-e547-4c48-be7f-9193caa535a3-kube-api-access-vgcb4\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.276927 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.276904    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d97d8c7-e547-4c48-be7f-9193caa535a3-openshift-service-ca\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.278793 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.278763    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d97d8c7-e547-4c48-be7f-9193caa535a3-maas-api-tls\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.289020 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.288994    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgcb4\" (UniqueName: \"kubernetes.io/projected/8d97d8c7-e547-4c48-be7f-9193caa535a3-kube-api-access-vgcb4\") pod \"maas-api-e2e-derive-70108423-b75574859-9cbnr\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") " pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.371113 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.371078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:05.506615 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.506590    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"]
Jul 07 00:22:05.508376 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:22:05.508347    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d97d8c7_e547_4c48_be7f_9193caa535a3.slice/crio-3dccb289378b3a50ca87178b4ad8bcee5f15fd5cb212897c9221466750ff3143 WatchSource:0}: Error finding container 3dccb289378b3a50ca87178b4ad8bcee5f15fd5cb212897c9221466750ff3143: Status 404 returned error can't find the container with id 3dccb289378b3a50ca87178b4ad8bcee5f15fd5cb212897c9221466750ff3143
Jul 07 00:22:05.593969 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.593935    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr" event={"ID":"8d97d8c7-e547-4c48-be7f-9193caa535a3","Type":"ContainerStarted","Data":"3dccb289378b3a50ca87178b4ad8bcee5f15fd5cb212897c9221466750ff3143"}
Jul 07 00:22:05.594246 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.594220    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:05.595116 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:05.595101    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:06.600433 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:06.600400    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr" event={"ID":"8d97d8c7-e547-4c48-be7f-9193caa535a3","Type":"ContainerStarted","Data":"fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca"}
Jul 07 00:22:06.600840 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:06.600648    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:06.644359 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:06.644302    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr" podStartSLOduration=1.196730632 podStartE2EDuration="1.644284966s" totalImagesPullingTime="447.554334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:22:06.641968786 +0000 UTC m=+1559.054982717" watchObservedRunningTime="2026-07-07 00:22:06.644284966 +0000 UTC m=+1559.057298871"
Jul 07 00:22:09.491586 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:09.491546    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"]
Jul 07 00:22:09.492083 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:09.491793    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws" podUID="3f38900b-b6f4-4ad7-993e-379e1d22cc33" containerName="istio-proxy" containerID="cri-o://42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b" gracePeriod=30
Jul 07 00:22:09.723521 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:09.723491    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"]
Jul 07 00:22:09.724340 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:09.724308    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr" podUID="8d97d8c7-e547-4c48-be7f-9193caa535a3" containerName="maas-api" containerID="cri-o://fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca" gracePeriod=30
Jul 07 00:22:10.016091 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.016009    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:10.123542 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.123515    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d97d8c7-e547-4c48-be7f-9193caa535a3-maas-api-tls\") pod \"8d97d8c7-e547-4c48-be7f-9193caa535a3\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") "
Jul 07 00:22:10.123729 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.123553    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgcb4\" (UniqueName: \"kubernetes.io/projected/8d97d8c7-e547-4c48-be7f-9193caa535a3-kube-api-access-vgcb4\") pod \"8d97d8c7-e547-4c48-be7f-9193caa535a3\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") "
Jul 07 00:22:10.123729 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.123596    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d97d8c7-e547-4c48-be7f-9193caa535a3-openshift-service-ca\") pod \"8d97d8c7-e547-4c48-be7f-9193caa535a3\" (UID: \"8d97d8c7-e547-4c48-be7f-9193caa535a3\") "
Jul 07 00:22:10.124228 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.124200    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d97d8c7-e547-4c48-be7f-9193caa535a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d97d8c7-e547-4c48-be7f-9193caa535a3" (UID: "8d97d8c7-e547-4c48-be7f-9193caa535a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:22:10.125848 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.125825    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d97d8c7-e547-4c48-be7f-9193caa535a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d97d8c7-e547-4c48-be7f-9193caa535a3" (UID: "8d97d8c7-e547-4c48-be7f-9193caa535a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:22:10.125922 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.125841    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d97d8c7-e547-4c48-be7f-9193caa535a3-kube-api-access-vgcb4" (OuterVolumeSpecName: "kube-api-access-vgcb4") pod "8d97d8c7-e547-4c48-be7f-9193caa535a3" (UID: "8d97d8c7-e547-4c48-be7f-9193caa535a3"). InnerVolumeSpecName "kube-api-access-vgcb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:22:10.224571 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.224539    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d97d8c7-e547-4c48-be7f-9193caa535a3-maas-api-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:10.224571 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.224567    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgcb4\" (UniqueName: \"kubernetes.io/projected/8d97d8c7-e547-4c48-be7f-9193caa535a3-kube-api-access-vgcb4\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:10.224571 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.224577    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d97d8c7-e547-4c48-be7f-9193caa535a3-openshift-service-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:10.621622 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.621580    2564 generic.go:356] "Generic (PLEG): container finished" podID="8d97d8c7-e547-4c48-be7f-9193caa535a3" containerID="fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca" exitCode=0
Jul 07 00:22:10.622031 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.621631    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"
Jul 07 00:22:10.622031 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.621664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr" event={"ID":"8d97d8c7-e547-4c48-be7f-9193caa535a3","Type":"ContainerDied","Data":"fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca"}
Jul 07 00:22:10.622031 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.621720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr" event={"ID":"8d97d8c7-e547-4c48-be7f-9193caa535a3","Type":"ContainerDied","Data":"3dccb289378b3a50ca87178b4ad8bcee5f15fd5cb212897c9221466750ff3143"}
Jul 07 00:22:10.622031 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.621742    2564 scope.go:117] "RemoveContainer" containerID="fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca"
Jul 07 00:22:10.655029 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.654982    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"]
Jul 07 00:22:10.655178 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.655137    2564 scope.go:117] "RemoveContainer" containerID="fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca"
Jul 07 00:22:10.655561 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:22:10.655524    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca\": container with ID starting with fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca not found: ID does not exist" containerID="fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca"
Jul 07 00:22:10.655681 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.655565    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca"} err="failed to get container status \"fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca\": rpc error: code = NotFound desc = could not find container \"fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca\": container with ID starting with fbe7e98b02f802ff1cef32b4f5638f3e73f186291f7c5b9366df9643440414ca not found: ID does not exist"
Jul 07 00:22:10.659662 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:10.659642    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-70108423-b75574859-9cbnr"]
Jul 07 00:22:12.115471 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:12.115433    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d97d8c7-e547-4c48-be7f-9193caa535a3" path="/var/lib/kubelet/pods/8d97d8c7-e547-4c48-be7f-9193caa535a3/volumes"
Jul 07 00:22:15.638754 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.638730    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:15.648831 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.648807    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f38900b-b6f4-4ad7-993e-379e1d22cc33" containerID="42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b" exitCode=0
Jul 07 00:22:15.648931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.648845    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"
Jul 07 00:22:15.648931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.648845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws" event={"ID":"3f38900b-b6f4-4ad7-993e-379e1d22cc33","Type":"ContainerDied","Data":"42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b"}
Jul 07 00:22:15.649004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.648957    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws" event={"ID":"3f38900b-b6f4-4ad7-993e-379e1d22cc33","Type":"ContainerDied","Data":"fdf6c7570ef85ba1cf65ded95fc68385a10c53e2ed1be257015378a392965758"}
Jul 07 00:22:15.649004 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.648984    2564 scope.go:117] "RemoveContainer" containerID="42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b"
Jul 07 00:22:15.657109 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.657092    2564 scope.go:117] "RemoveContainer" containerID="42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b"
Jul 07 00:22:15.657341 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:22:15.657324    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b\": container with ID starting with 42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b not found: ID does not exist" containerID="42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b"
Jul 07 00:22:15.657383 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.657355    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b"} err="failed to get container status \"42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b\": rpc error: code = NotFound desc = could not find container \"42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b\": container with ID starting with 42801d8128bee6b28492a0ac8ade8c5c14802f5557e710ff347d561c91f74e9b not found: ID does not exist"
Jul 07 00:22:15.785875 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.785784    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istiod-ca-cert\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.785875 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.785832    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-socket\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.785880    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-certs\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.785935    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-envoy\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.785960    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-data\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.785995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-token\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786037    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-credential-socket\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786083    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k9pr\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-kube-api-access-9k9pr\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786112 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786109    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-podinfo\") pod \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\" (UID: \"3f38900b-b6f4-4ad7-993e-379e1d22cc33\") "
Jul 07 00:22:15.786520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786192    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:22:15.786520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786286    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:22:15.786520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786323    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:22:15.786520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786437    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-credential-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.786520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786484    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istiod-ca-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.786520 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786500    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.786829 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786666    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:22:15.786871 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.786832    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:22:15.788195 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.788162    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:22:15.788332 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.788287    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-kube-api-access-9k9pr" (OuterVolumeSpecName: "kube-api-access-9k9pr") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "kube-api-access-9k9pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:22:15.788529 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.788501    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:22:15.788633 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.788550    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f38900b-b6f4-4ad7-993e-379e1d22cc33" (UID: "3f38900b-b6f4-4ad7-993e-379e1d22cc33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:22:15.887925 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.887895    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k9pr\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-kube-api-access-9k9pr\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.887925 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.887920    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-podinfo\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.887925 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.887929    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-workload-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.888130 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.887937    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-envoy\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.888130 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.887945    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-data\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.888130 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.887952    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f38900b-b6f4-4ad7-993e-379e1d22cc33-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:22:15.972091 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.972065    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"]
Jul 07 00:22:15.976907 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:15.976883    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-70108423-openshift-default-75798568d7-5vsws"]
Jul 07 00:22:16.114693 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:22:16.114657    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f38900b-b6f4-4ad7-993e-379e1d22cc33" path="/var/lib/kubelet/pods/3f38900b-b6f4-4ad7-993e-379e1d22cc33/volumes"
Jul 07 00:23:52.404604 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.404570    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"]
Jul 07 00:23:52.405172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.404993    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f38900b-b6f4-4ad7-993e-379e1d22cc33" containerName="istio-proxy"
Jul 07 00:23:52.405172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.405006    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f38900b-b6f4-4ad7-993e-379e1d22cc33" containerName="istio-proxy"
Jul 07 00:23:52.405172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.405025    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d97d8c7-e547-4c48-be7f-9193caa535a3" containerName="maas-api"
Jul 07 00:23:52.405172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.405030    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d97d8c7-e547-4c48-be7f-9193caa535a3" containerName="maas-api"
Jul 07 00:23:52.405172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.405086    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f38900b-b6f4-4ad7-993e-379e1d22cc33" containerName="istio-proxy"
Jul 07 00:23:52.405172 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.405095    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d97d8c7-e547-4c48-be7f-9193caa535a3" containerName="maas-api"
Jul 07 00:23:52.408293 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.408277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.411560 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.411537    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-22d66e0c-openshift-default-dockercfg-lf82s\""
Jul 07 00:23:52.434084 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.434047    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"]
Jul 07 00:23:52.555609 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-credential-socket\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555609 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-socket\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-data\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd2q4\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-kube-api-access-dd2q4\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-token\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a58c53f7-2199-495c-9920-c06a9880cf0d-istiod-ca-cert\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-certs\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.555873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555797    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-podinfo\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.556212 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.555905    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-envoy\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-socket\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-data\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd2q4\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-kube-api-access-dd2q4\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-token\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a58c53f7-2199-495c-9920-c06a9880cf0d-istiod-ca-cert\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-certs\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-podinfo\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-envoy\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657635    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-socket\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-credential-socket\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657761 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657739    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-data\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657924 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657817    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-certs\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.657961 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.657939    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a58c53f7-2199-495c-9920-c06a9880cf0d-istiod-ca-cert\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.658043 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.658021    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-credential-socket\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.659747 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.659722    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-envoy\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.659906 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.659889    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-podinfo\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.665844 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.665818    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-token\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.665988 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.665897    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd2q4\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-kube-api-access-dd2q4\") pod \"e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") " pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.718269 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.718228    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:52.873494 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.871343    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"]
Jul 07 00:23:52.875251 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:23:52.875217    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58c53f7_2199_495c_9920_c06a9880cf0d.slice/crio-b0e0d58b8fd5e9cf540a5ceae4c69c8a5ae93a249b100b4743ecae5dc78d5b9d WatchSource:0}: Error finding container b0e0d58b8fd5e9cf540a5ceae4c69c8a5ae93a249b100b4743ecae5dc78d5b9d: Status 404 returned error can't find the container with id b0e0d58b8fd5e9cf540a5ceae4c69c8a5ae93a249b100b4743ecae5dc78d5b9d
Jul 07 00:23:52.877708 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.877669    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:23:52.877786 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.877755    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:23:52.877825 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:52.877798    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:23:53.125695 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:53.125656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" event={"ID":"a58c53f7-2199-495c-9920-c06a9880cf0d","Type":"ContainerStarted","Data":"be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"}
Jul 07 00:23:53.125873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:53.125725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" event={"ID":"a58c53f7-2199-495c-9920-c06a9880cf0d","Type":"ContainerStarted","Data":"b0e0d58b8fd5e9cf540a5ceae4c69c8a5ae93a249b100b4743ecae5dc78d5b9d"}
Jul 07 00:23:53.149804 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:53.149751    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" podStartSLOduration=1.149733347 podStartE2EDuration="1.149733347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:53.148835792 +0000 UTC m=+1665.561849700" watchObservedRunningTime="2026-07-07 00:23:53.149733347 +0000 UTC m=+1665.562747257"
Jul 07 00:23:53.718987 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:53.718948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:53.724398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:53.724371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:54.131245 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:54.131208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:54.132154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:54.132135    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:23:55.385631 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.385596    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"]
Jul 07 00:23:55.389619 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.389593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.392320 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.392288    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-22d66e0c-serving-cert\""
Jul 07 00:23:55.401914 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.401889    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"]
Jul 07 00:23:55.488782 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.488727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f89ae1-b84f-4101-a172-581a5c00e9a3-openshift-service-ca\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.489032 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.488868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whbcc\" (UniqueName: \"kubernetes.io/projected/e1f89ae1-b84f-4101-a172-581a5c00e9a3-kube-api-access-whbcc\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.489032 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.488949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.590196 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.590157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.590427 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.590225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f89ae1-b84f-4101-a172-581a5c00e9a3-openshift-service-ca\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.590427 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:23:55.590317    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-22d66e0c-serving-cert: secret "maas-api-e2e-mt-22d66e0c-serving-cert" not found
Jul 07 00:23:55.590427 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.590319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whbcc\" (UniqueName: \"kubernetes.io/projected/e1f89ae1-b84f-4101-a172-581a5c00e9a3-kube-api-access-whbcc\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.590427 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:23:55.590396    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls podName:e1f89ae1-b84f-4101-a172-581a5c00e9a3 nodeName:}" failed. No retries permitted until 2026-07-07 00:23:56.09037808 +0000 UTC m=+1668.503391965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls") pod "maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" (UID: "e1f89ae1-b84f-4101-a172-581a5c00e9a3") : secret "maas-api-e2e-mt-22d66e0c-serving-cert" not found
Jul 07 00:23:55.591033 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.591012    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f89ae1-b84f-4101-a172-581a5c00e9a3-openshift-service-ca\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:55.607011 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:55.606975    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whbcc\" (UniqueName: \"kubernetes.io/projected/e1f89ae1-b84f-4101-a172-581a5c00e9a3-kube-api-access-whbcc\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:56.095690 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:56.095651    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:56.098327 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:56.098297    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls\") pod \"maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") " pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:56.304016 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:56.303986    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:56.446737 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:56.446706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"]
Jul 07 00:23:56.447979 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:23:56.447957    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f89ae1_b84f_4101_a172_581a5c00e9a3.slice/crio-3c6ed948c8eb6e11922765f0746f5b6d73f5fc502562f43b5c46dec9dca4fb19 WatchSource:0}: Error finding container 3c6ed948c8eb6e11922765f0746f5b6d73f5fc502562f43b5c46dec9dca4fb19: Status 404 returned error can't find the container with id 3c6ed948c8eb6e11922765f0746f5b6d73f5fc502562f43b5c46dec9dca4fb19
Jul 07 00:23:57.147674 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:57.147580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" event={"ID":"e1f89ae1-b84f-4101-a172-581a5c00e9a3","Type":"ContainerStarted","Data":"bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f"}
Jul 07 00:23:57.147674 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:57.147636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" event={"ID":"e1f89ae1-b84f-4101-a172-581a5c00e9a3","Type":"ContainerStarted","Data":"3c6ed948c8eb6e11922765f0746f5b6d73f5fc502562f43b5c46dec9dca4fb19"}
Jul 07 00:23:57.147933 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:57.147686    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:23:57.166127 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:23:57.166075    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" podStartSLOduration=1.748239392 podStartE2EDuration="2.166058477s" totalImagesPullingTime="417.819085ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:57.163868866 +0000 UTC m=+1669.576882793" watchObservedRunningTime="2026-07-07 00:23:57.166058477 +0000 UTC m=+1669.579072382"
Jul 07 00:24:03.159400 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:03.159369    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:24:06.197830 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:06.197797    2564 generic.go:356] "Generic (PLEG): container finished" podID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerID="be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953" exitCode=0
Jul 07 00:24:06.198295 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:06.197867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" event={"ID":"a58c53f7-2199-495c-9920-c06a9880cf0d","Type":"ContainerDied","Data":"be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"}
Jul 07 00:24:06.198295 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:06.198243    2564 scope.go:117] "RemoveContainer" containerID="be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"
Jul 07 00:24:06.199296 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:06.199272    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:24:06.199358 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:06.199345    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:24:06.199397 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:06.199381    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:24:07.205310 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:07.205280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" event={"ID":"a58c53f7-2199-495c-9920-c06a9880cf0d","Type":"ContainerStarted","Data":"eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a"}
Jul 07 00:24:07.718727 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:07.718691    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:24:07.719908 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:07.719886    2564 patch_prober.go:28] interesting pod/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 07 00:24:07.720017 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:07.719941    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 07 00:24:07.854544 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:07.849133    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"]
Jul 07 00:24:07.854544 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:07.849372    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" podUID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" containerName="maas-api" containerID="cri-o://bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f" gracePeriod=30
Jul 07 00:24:08.167551 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.167523    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:24:08.211731 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.211694    2564 generic.go:356] "Generic (PLEG): container finished" podID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" containerID="bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f" exitCode=0
Jul 07 00:24:08.212196 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.211736    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" event={"ID":"e1f89ae1-b84f-4101-a172-581a5c00e9a3","Type":"ContainerDied","Data":"bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f"}
Jul 07 00:24:08.212196 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.211756    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"
Jul 07 00:24:08.212196 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.211802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" event={"ID":"e1f89ae1-b84f-4101-a172-581a5c00e9a3","Type":"ContainerDied","Data":"3c6ed948c8eb6e11922765f0746f5b6d73f5fc502562f43b5c46dec9dca4fb19"}
Jul 07 00:24:08.212196 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.211834    2564 scope.go:117] "RemoveContainer" containerID="bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f"
Jul 07 00:24:08.212743 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.212722    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls\") pod \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") "
Jul 07 00:24:08.212847 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.212760    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f89ae1-b84f-4101-a172-581a5c00e9a3-openshift-service-ca\") pod \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") "
Jul 07 00:24:08.212847 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.212840    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whbcc\" (UniqueName: \"kubernetes.io/projected/e1f89ae1-b84f-4101-a172-581a5c00e9a3-kube-api-access-whbcc\") pod \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\" (UID: \"e1f89ae1-b84f-4101-a172-581a5c00e9a3\") "
Jul 07 00:24:08.213238 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.213212    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1f89ae1-b84f-4101-a172-581a5c00e9a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1f89ae1-b84f-4101-a172-581a5c00e9a3" (UID: "e1f89ae1-b84f-4101-a172-581a5c00e9a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:24:08.214902 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.214879    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1f89ae1-b84f-4101-a172-581a5c00e9a3" (UID: "e1f89ae1-b84f-4101-a172-581a5c00e9a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:24:08.215363 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.215337    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1f89ae1-b84f-4101-a172-581a5c00e9a3-kube-api-access-whbcc" (OuterVolumeSpecName: "kube-api-access-whbcc") pod "e1f89ae1-b84f-4101-a172-581a5c00e9a3" (UID: "e1f89ae1-b84f-4101-a172-581a5c00e9a3"). InnerVolumeSpecName "kube-api-access-whbcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:08.257811 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.257742    2564 scope.go:117] "RemoveContainer" containerID="bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f"
Jul 07 00:24:08.258078 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:24:08.258041    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f\": container with ID starting with bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f not found: ID does not exist" containerID="bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f"
Jul 07 00:24:08.258152 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.258090    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f"} err="failed to get container status \"bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f\": rpc error: code = NotFound desc = could not find container \"bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f\": container with ID starting with bbd3cc3148cc2db7fd86b3c7e602e3465d81712ff04ab597d221f5808767af6f not found: ID does not exist"
Jul 07 00:24:08.314060 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.314023    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whbcc\" (UniqueName: \"kubernetes.io/projected/e1f89ae1-b84f-4101-a172-581a5c00e9a3-kube-api-access-whbcc\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.314060 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.314054    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f89ae1-b84f-4101-a172-581a5c00e9a3-maas-api-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.314060 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.314063    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f89ae1-b84f-4101-a172-581a5c00e9a3-openshift-service-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:08.539485 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.536621    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"]
Jul 07 00:24:08.550337 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.550297    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z"]
Jul 07 00:24:08.718831 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.718800    2564 patch_prober.go:28] interesting pod/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 07 00:24:08.718982 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:08.718852    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 07 00:24:09.131386 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:09.131331    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:24:09.719011 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:09.718973    2564 patch_prober.go:28] interesting pod/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 07 00:24:09.719415 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:09.719033    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 07 00:24:10.120873 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:10.120836    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" path="/var/lib/kubelet/pods/e1f89ae1-b84f-4101-a172-581a5c00e9a3/volumes"
Jul 07 00:24:10.718922 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:10.718885    2564 patch_prober.go:28] interesting pod/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 07 00:24:10.719103 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:10.718947    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 07 00:24:11.158401 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:11.158348    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-22d66e0c-5f9f49b9b8-rkl2z" podUID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.58:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:24:11.722696 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:11.722663    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:24:11.723777 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:11.723755    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:24:14.003835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:14.003802    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"]
Jul 07 00:24:14.004292 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:14.004040    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy" containerID="cri-o://eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a" gracePeriod=30
Jul 07 00:24:19.263937 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.263879    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:24:19.271089 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.271057    2564 generic.go:356] "Generic (PLEG): container finished" podID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerID="eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a" exitCode=0
Jul 07 00:24:19.271214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.271103    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"
Jul 07 00:24:19.271214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.271134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" event={"ID":"a58c53f7-2199-495c-9920-c06a9880cf0d","Type":"ContainerDied","Data":"eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a"}
Jul 07 00:24:19.271214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.271190    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8" event={"ID":"a58c53f7-2199-495c-9920-c06a9880cf0d","Type":"ContainerDied","Data":"b0e0d58b8fd5e9cf540a5ceae4c69c8a5ae93a249b100b4743ecae5dc78d5b9d"}
Jul 07 00:24:19.271214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.271212    2564 scope.go:117] "RemoveContainer" containerID="eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a"
Jul 07 00:24:19.280401 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.280384    2564 scope.go:117] "RemoveContainer" containerID="be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"
Jul 07 00:24:19.288239 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.288220    2564 scope.go:117] "RemoveContainer" containerID="eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a"
Jul 07 00:24:19.288499 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:24:19.288478    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a\": container with ID starting with eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a not found: ID does not exist" containerID="eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a"
Jul 07 00:24:19.288622 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.288512    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a"} err="failed to get container status \"eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a\": rpc error: code = NotFound desc = could not find container \"eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a\": container with ID starting with eaea3b48b7dddec32dd5f6a4cff466d9d041365c3543025be925d04c17c7d96a not found: ID does not exist"
Jul 07 00:24:19.288622 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.288536    2564 scope.go:117] "RemoveContainer" containerID="be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"
Jul 07 00:24:19.288833 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:24:19.288806    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953\": container with ID starting with be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953 not found: ID does not exist" containerID="be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"
Jul 07 00:24:19.288877 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.288839    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953"} err="failed to get container status \"be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953\": rpc error: code = NotFound desc = could not find container \"be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953\": container with ID starting with be703fe787ec2ecde157c53fa4902762fe44fb1b40c5131f709f99c9d7965953 not found: ID does not exist"
Jul 07 00:24:19.424302 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424269    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-socket\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424324    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-token\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424365    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a58c53f7-2199-495c-9920-c06a9880cf0d-istiod-ca-cert\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424385    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-certs\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424446    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-credential-socket\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424539 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424513    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-podinfo\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424556    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-data\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424594    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:19.424808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424609    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd2q4\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-kube-api-access-dd2q4\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424668    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-envoy\") pod \"a58c53f7-2199-495c-9920-c06a9880cf0d\" (UID: \"a58c53f7-2199-495c-9920-c06a9880cf0d\") "
Jul 07 00:24:19.424808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424760    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:19.425126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424812    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:19.425126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.424843    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58c53f7-2199-495c-9920-c06a9880cf0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:24:19.425126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.425029    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-credential-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.425126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.425050    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.425126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.425064    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a58c53f7-2199-495c-9920-c06a9880cf0d-istiod-ca-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.425126 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.425077    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-workload-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.425541 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.425523    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:19.427188 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.427160    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:24:19.427302 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.427200    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:19.427302 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.427292    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:19.427381 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.427314    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-kube-api-access-dd2q4" (OuterVolumeSpecName: "kube-api-access-dd2q4") pod "a58c53f7-2199-495c-9920-c06a9880cf0d" (UID: "a58c53f7-2199-495c-9920-c06a9880cf0d"). InnerVolumeSpecName "kube-api-access-dd2q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:19.526364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.526278    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.526364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.526308    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-podinfo\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.526364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.526317    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-data\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.526364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.526327    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd2q4\" (UniqueName: \"kubernetes.io/projected/a58c53f7-2199-495c-9920-c06a9880cf0d-kube-api-access-dd2q4\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.526364 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.526336    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a58c53f7-2199-495c-9920-c06a9880cf0d-istio-envoy\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:24:19.593706 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.593677    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"]
Jul 07 00:24:19.598730 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:19.598703    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-22d66e0c-openshift-default-88fc85c4c-pr2n8"]
Jul 07 00:24:20.115625 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:20.115596    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" path="/var/lib/kubelet/pods/a58c53f7-2199-495c-9920-c06a9880cf0d/volumes"
Jul 07 00:24:36.626158 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626072    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"]
Jul 07 00:24:36.626758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626616    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" containerName="maas-api"
Jul 07 00:24:36.626758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626644    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" containerName="maas-api"
Jul 07 00:24:36.626758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626661    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy"
Jul 07 00:24:36.626758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626666    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy"
Jul 07 00:24:36.626758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626760    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy"
Jul 07 00:24:36.627052 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626774    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy"
Jul 07 00:24:36.627052 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.626785    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1f89ae1-b84f-4101-a172-581a5c00e9a3" containerName="maas-api"
Jul 07 00:24:36.631530 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.631512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.634255 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.634235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-0dd24d04-serving-cert\""
Jul 07 00:24:36.643253 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.643231    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"]
Jul 07 00:24:36.678498 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.678441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.678639 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.678553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2829344d-c094-409a-a2c0-18ab497b8fec-openshift-service-ca\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.678639 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.678602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slb7t\" (UniqueName: \"kubernetes.io/projected/2829344d-c094-409a-a2c0-18ab497b8fec-kube-api-access-slb7t\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.779773 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.779737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slb7t\" (UniqueName: \"kubernetes.io/projected/2829344d-c094-409a-a2c0-18ab497b8fec-kube-api-access-slb7t\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.779946 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.779796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.779946 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.779854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2829344d-c094-409a-a2c0-18ab497b8fec-openshift-service-ca\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.780077 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:24:36.779968    2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-0dd24d04-serving-cert: secret "maas-api-e2e-mt-0dd24d04-serving-cert" not found
Jul 07 00:24:36.780077 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:24:36.780052    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls podName:2829344d-c094-409a-a2c0-18ab497b8fec nodeName:}" failed. No retries permitted until 2026-07-07 00:24:37.280030832 +0000 UTC m=+1709.693044729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls") pod "maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" (UID: "2829344d-c094-409a-a2c0-18ab497b8fec") : secret "maas-api-e2e-mt-0dd24d04-serving-cert" not found
Jul 07 00:24:36.780540 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.780522    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2829344d-c094-409a-a2c0-18ab497b8fec-openshift-service-ca\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:36.790311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:36.790288    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slb7t\" (UniqueName: \"kubernetes.io/projected/2829344d-c094-409a-a2c0-18ab497b8fec-kube-api-access-slb7t\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:37.283934 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:37.283904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:37.286227 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:37.286200    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls\") pod \"maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") " pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:37.543633 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:37.543549    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:37.696528 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:37.696503    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"]
Jul 07 00:24:37.698349 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:24:37.698322    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2829344d_c094_409a_a2c0_18ab497b8fec.slice/crio-4fab0e6f3cc058a39bfca4bbd6f8703923477f2ca9a6220c4aa9d701a29f7425 WatchSource:0}: Error finding container 4fab0e6f3cc058a39bfca4bbd6f8703923477f2ca9a6220c4aa9d701a29f7425: Status 404 returned error can't find the container with id 4fab0e6f3cc058a39bfca4bbd6f8703923477f2ca9a6220c4aa9d701a29f7425
Jul 07 00:24:38.371995 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:38.371960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" event={"ID":"2829344d-c094-409a-a2c0-18ab497b8fec","Type":"ContainerStarted","Data":"4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0"}
Jul 07 00:24:38.372182 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:38.372011    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" event={"ID":"2829344d-c094-409a-a2c0-18ab497b8fec","Type":"ContainerStarted","Data":"4fab0e6f3cc058a39bfca4bbd6f8703923477f2ca9a6220c4aa9d701a29f7425"}
Jul 07 00:24:38.372182 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:38.372079    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:24:38.396414 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:38.396315    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" podStartSLOduration=1.9558900289999999 podStartE2EDuration="2.396296843s" totalImagesPullingTime="440.406814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:38.394891212 +0000 UTC m=+1710.807905136" watchObservedRunningTime="2026-07-07 00:24:38.396296843 +0000 UTC m=+1710.809310752"
Jul 07 00:24:44.383386 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:24:44.383357    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:25:08.678035 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:08.675134    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"]
Jul 07 00:25:08.678035 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:08.675468    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" podUID="2829344d-c094-409a-a2c0-18ab497b8fec" containerName="maas-api" containerID="cri-o://4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0" gracePeriod=30
Jul 07 00:25:08.935573 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:08.935511    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:25:09.061184 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.061155    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2829344d-c094-409a-a2c0-18ab497b8fec-openshift-service-ca\") pod \"2829344d-c094-409a-a2c0-18ab497b8fec\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") "
Jul 07 00:25:09.061361 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.061217    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls\") pod \"2829344d-c094-409a-a2c0-18ab497b8fec\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") "
Jul 07 00:25:09.061361 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.061235    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slb7t\" (UniqueName: \"kubernetes.io/projected/2829344d-c094-409a-a2c0-18ab497b8fec-kube-api-access-slb7t\") pod \"2829344d-c094-409a-a2c0-18ab497b8fec\" (UID: \"2829344d-c094-409a-a2c0-18ab497b8fec\") "
Jul 07 00:25:09.061616 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.061589    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2829344d-c094-409a-a2c0-18ab497b8fec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2829344d-c094-409a-a2c0-18ab497b8fec" (UID: "2829344d-c094-409a-a2c0-18ab497b8fec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:25:09.063323 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.063300    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2829344d-c094-409a-a2c0-18ab497b8fec-kube-api-access-slb7t" (OuterVolumeSpecName: "kube-api-access-slb7t") pod "2829344d-c094-409a-a2c0-18ab497b8fec" (UID: "2829344d-c094-409a-a2c0-18ab497b8fec"). InnerVolumeSpecName "kube-api-access-slb7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:25:09.063396 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.063374    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2829344d-c094-409a-a2c0-18ab497b8fec" (UID: "2829344d-c094-409a-a2c0-18ab497b8fec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:25:09.162904 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.162875    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2829344d-c094-409a-a2c0-18ab497b8fec-openshift-service-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:25:09.162904 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.162905    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2829344d-c094-409a-a2c0-18ab497b8fec-maas-api-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:25:09.163091 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.162920    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slb7t\" (UniqueName: \"kubernetes.io/projected/2829344d-c094-409a-a2c0-18ab497b8fec-kube-api-access-slb7t\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:25:09.533562 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.533521    2564 generic.go:356] "Generic (PLEG): container finished" podID="2829344d-c094-409a-a2c0-18ab497b8fec" containerID="4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0" exitCode=0
Jul 07 00:25:09.533562 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.533560    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"
Jul 07 00:25:09.533956 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.533606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" event={"ID":"2829344d-c094-409a-a2c0-18ab497b8fec","Type":"ContainerDied","Data":"4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0"}
Jul 07 00:25:09.533956 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.533662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd" event={"ID":"2829344d-c094-409a-a2c0-18ab497b8fec","Type":"ContainerDied","Data":"4fab0e6f3cc058a39bfca4bbd6f8703923477f2ca9a6220c4aa9d701a29f7425"}
Jul 07 00:25:09.533956 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.533689    2564 scope.go:117] "RemoveContainer" containerID="4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0"
Jul 07 00:25:09.543221 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.543204    2564 scope.go:117] "RemoveContainer" containerID="4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0"
Jul 07 00:25:09.543489 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:25:09.543469    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0\": container with ID starting with 4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0 not found: ID does not exist" containerID="4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0"
Jul 07 00:25:09.543569 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.543502    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0"} err="failed to get container status \"4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0\": rpc error: code = NotFound desc = could not find container \"4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0\": container with ID starting with 4f6aecedcc81bedf710b047949cf0c8fb572e49f5ac599c33b205a5a965331c0 not found: ID does not exist"
Jul 07 00:25:09.555981 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.555961    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"]
Jul 07 00:25:09.559830 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:09.559811    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-0dd24d04-7ddf585c64-7c4nd"]
Jul 07 00:25:10.115051 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:25:10.115019    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2829344d-c094-409a-a2c0-18ab497b8fec" path="/var/lib/kubelet/pods/2829344d-c094-409a-a2c0-18ab497b8fec/volumes"
Jul 07 00:26:08.176621 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:08.176593    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:26:08.176621 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:08.176611    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:26:58.360510 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.360478    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"]
Jul 07 00:26:58.360960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.360860    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy"
Jul 07 00:26:58.360960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.360872    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a58c53f7-2199-495c-9920-c06a9880cf0d" containerName="istio-proxy"
Jul 07 00:26:58.360960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.360888    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2829344d-c094-409a-a2c0-18ab497b8fec" containerName="maas-api"
Jul 07 00:26:58.360960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.360894    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2829344d-c094-409a-a2c0-18ab497b8fec" containerName="maas-api"
Jul 07 00:26:58.360960 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.360950    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2829344d-c094-409a-a2c0-18ab497b8fec" containerName="maas-api"
Jul 07 00:26:58.364109 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.364089    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.366842 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.366824    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7eba6c-openshift-default-dockercfg-r7mzp\""
Jul 07 00:26:58.378577 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.378556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"]
Jul 07 00:26:58.521078 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-data\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521304 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed1130b-cd51-460b-a5b7-d5f4788565da-istiod-ca-cert\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521304 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-credential-socket\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521304 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-token\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521304 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-socket\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521488 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521331    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-certs\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521488 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-podinfo\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521488 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2flmr\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-kube-api-access-2flmr\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.521488 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.521437    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-envoy\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622249 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-credential-socket\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622249 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622202    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-token\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622503 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-socket\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622503 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-certs\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622503 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-podinfo\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622679 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2flmr\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-kube-api-access-2flmr\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622679 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-envoy\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622679 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-data\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622679 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622652    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-credential-socket\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622679 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622659    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed1130b-cd51-460b-a5b7-d5f4788565da-istiod-ca-cert\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622947 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622740    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-certs\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.622947 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.622853    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-socket\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.623056 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.623007    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-data\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.623240 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.623224    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed1130b-cd51-460b-a5b7-d5f4788565da-istiod-ca-cert\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.624650 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.624628    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-envoy\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.624843 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.624826    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-podinfo\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.630555 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.630535    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2flmr\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-kube-api-access-2flmr\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.630691 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.630675    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-token\") pod \"e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") " pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.674215 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.674189    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:58.812473 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.812434    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"]
Jul 07 00:26:58.814059 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:26:58.814034    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed1130b_cd51_460b_a5b7_d5f4788565da.slice/crio-1cde4f3ccbd75db0a6e858f9535cd796d8975b0bdad07d50886899b4035cab4f WatchSource:0}: Error finding container 1cde4f3ccbd75db0a6e858f9535cd796d8975b0bdad07d50886899b4035cab4f: Status 404 returned error can't find the container with id 1cde4f3ccbd75db0a6e858f9535cd796d8975b0bdad07d50886899b4035cab4f
Jul 07 00:26:58.816197 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.816168    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:26:58.816283 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.816233    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:26:58.816283 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:58.816258    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:26:59.063684 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:59.063654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" event={"ID":"fed1130b-cd51-460b-a5b7-d5f4788565da","Type":"ContainerStarted","Data":"2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"}
Jul 07 00:26:59.063862 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:59.063702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" event={"ID":"fed1130b-cd51-460b-a5b7-d5f4788565da","Type":"ContainerStarted","Data":"1cde4f3ccbd75db0a6e858f9535cd796d8975b0bdad07d50886899b4035cab4f"}
Jul 07 00:26:59.100946 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:59.100898    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podStartSLOduration=1.100880736 podStartE2EDuration="1.100880736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:59.098786963 +0000 UTC m=+1851.511800871" watchObservedRunningTime="2026-07-07 00:26:59.100880736 +0000 UTC m=+1851.513894647"
Jul 07 00:26:59.675347 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:59.675317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:26:59.680862 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:26:59.680833    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:27:00.069116 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:00.069031    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:27:00.069903 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:00.069885    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:27:01.084291 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.084260    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"]
Jul 07 00:27:01.088193 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.088168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.091778 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.091704    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-7eba6c-serving-cert\""
Jul 07 00:27:01.101774 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.101742    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"]
Jul 07 00:27:01.247268 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.247237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/262a45bd-5a7a-44d5-bb39-188762c53aaa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.247445 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.247297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/262a45bd-5a7a-44d5-bb39-188762c53aaa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.247445 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.247392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9rnm\" (UniqueName: \"kubernetes.io/projected/262a45bd-5a7a-44d5-bb39-188762c53aaa-kube-api-access-n9rnm\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.348002 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.347961    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/262a45bd-5a7a-44d5-bb39-188762c53aaa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.348174 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.348024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/262a45bd-5a7a-44d5-bb39-188762c53aaa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.348174 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.348096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9rnm\" (UniqueName: \"kubernetes.io/projected/262a45bd-5a7a-44d5-bb39-188762c53aaa-kube-api-access-n9rnm\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.348627 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.348609    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/262a45bd-5a7a-44d5-bb39-188762c53aaa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.350407 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.350383    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/262a45bd-5a7a-44d5-bb39-188762c53aaa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.357160 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.357133    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9rnm\" (UniqueName: \"kubernetes.io/projected/262a45bd-5a7a-44d5-bb39-188762c53aaa-kube-api-access-n9rnm\") pod \"maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") " pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.400884 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.400853    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:01.544828 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.544804    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"]
Jul 07 00:27:01.549321 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:27:01.549278    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod262a45bd_5a7a_44d5_bb39_188762c53aaa.slice/crio-a0dddf43c4a9ff6b8eac9857d26ebc43821feb7e895298c199ed2e6353c03e43 WatchSource:0}: Error finding container a0dddf43c4a9ff6b8eac9857d26ebc43821feb7e895298c199ed2e6353c03e43: Status 404 returned error can't find the container with id a0dddf43c4a9ff6b8eac9857d26ebc43821feb7e895298c199ed2e6353c03e43
Jul 07 00:27:01.552596 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:01.552574    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:27:02.079720 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:02.079689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m" event={"ID":"262a45bd-5a7a-44d5-bb39-188762c53aaa","Type":"ContainerStarted","Data":"a0dddf43c4a9ff6b8eac9857d26ebc43821feb7e895298c199ed2e6353c03e43"}
Jul 07 00:27:03.086372 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:03.086333    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m" event={"ID":"262a45bd-5a7a-44d5-bb39-188762c53aaa","Type":"ContainerStarted","Data":"ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910"}
Jul 07 00:27:03.086786 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:03.086418    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:03.105866 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:03.105822    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m" podStartSLOduration=1.566846089 podStartE2EDuration="2.10580722s" totalImagesPullingTime="538.961131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:03.1045589 +0000 UTC m=+1855.517572842" watchObservedRunningTime="2026-07-07 00:27:03.10580722 +0000 UTC m=+1855.518821129"
Jul 07 00:27:09.095799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:09.095770    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:27:16.110621 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:16.110576    2564 patch_prober.go:28] interesting pod/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 00:27:16.110994 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:16.110646    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:27:18.164796 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:18.164762    2564 generic.go:356] "Generic (PLEG): container finished" podID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerID="2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867" exitCode=0
Jul 07 00:27:18.165208 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:18.164842    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" event={"ID":"fed1130b-cd51-460b-a5b7-d5f4788565da","Type":"ContainerDied","Data":"2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"}
Jul 07 00:27:18.165252 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:18.165221    2564 scope.go:117] "RemoveContainer" containerID="2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"
Jul 07 00:27:18.166365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:18.166329    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:27:18.166435 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:18.166422    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:27:18.166515 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:18.166496    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:27:19.171775 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:19.171743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" event={"ID":"fed1130b-cd51-460b-a5b7-d5f4788565da","Type":"ContainerStarted","Data":"047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7"}
Jul 07 00:27:19.675283 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:19.675252    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:27:19.675482 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:19.675364    2564 patch_prober.go:28] interesting pod/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:27:19.675482 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:19.675418    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:27:20.675384 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:20.675345    2564 patch_prober.go:28] interesting pod/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:27:20.675833 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:20.675416    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:27:21.674808 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:21.674771    2564 patch_prober.go:28] interesting pod/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:27:21.674963 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:21.674838    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:27:22.675210 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:22.675173    2564 patch_prober.go:28] interesting pod/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:27:22.675617 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:22.675240    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:27:23.678000 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:23.677968    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:27:23.678389 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:23.678232    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:27:23.678976 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:27:23.678961    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:28:24.441649 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:24.441611    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"]
Jul 07 00:28:24.442267 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:24.441845    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m" podUID="262a45bd-5a7a-44d5-bb39-188762c53aaa" containerName="maas-api" containerID="cri-o://ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910" gracePeriod=30
Jul 07 00:28:24.899829 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:24.899805    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:28:25.023543 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.023414    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9rnm\" (UniqueName: \"kubernetes.io/projected/262a45bd-5a7a-44d5-bb39-188762c53aaa-kube-api-access-n9rnm\") pod \"262a45bd-5a7a-44d5-bb39-188762c53aaa\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") "
Jul 07 00:28:25.023728 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.023562    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/262a45bd-5a7a-44d5-bb39-188762c53aaa-maas-api-tls\") pod \"262a45bd-5a7a-44d5-bb39-188762c53aaa\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") "
Jul 07 00:28:25.023728 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.023585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/262a45bd-5a7a-44d5-bb39-188762c53aaa-openshift-service-ca\") pod \"262a45bd-5a7a-44d5-bb39-188762c53aaa\" (UID: \"262a45bd-5a7a-44d5-bb39-188762c53aaa\") "
Jul 07 00:28:25.023998 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.023968    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/262a45bd-5a7a-44d5-bb39-188762c53aaa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "262a45bd-5a7a-44d5-bb39-188762c53aaa" (UID: "262a45bd-5a7a-44d5-bb39-188762c53aaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:28:25.025626 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.025598    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/262a45bd-5a7a-44d5-bb39-188762c53aaa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "262a45bd-5a7a-44d5-bb39-188762c53aaa" (UID: "262a45bd-5a7a-44d5-bb39-188762c53aaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:28:25.025718 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.025708    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262a45bd-5a7a-44d5-bb39-188762c53aaa-kube-api-access-n9rnm" (OuterVolumeSpecName: "kube-api-access-n9rnm") pod "262a45bd-5a7a-44d5-bb39-188762c53aaa" (UID: "262a45bd-5a7a-44d5-bb39-188762c53aaa"). InnerVolumeSpecName "kube-api-access-n9rnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:28:25.125042 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.125006    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9rnm\" (UniqueName: \"kubernetes.io/projected/262a45bd-5a7a-44d5-bb39-188762c53aaa-kube-api-access-n9rnm\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:28:25.125042 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.125036    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/262a45bd-5a7a-44d5-bb39-188762c53aaa-maas-api-tls\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:28:25.125042 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.125050    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/262a45bd-5a7a-44d5-bb39-188762c53aaa-openshift-service-ca\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:28:25.501651 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.501617    2564 generic.go:356] "Generic (PLEG): container finished" podID="262a45bd-5a7a-44d5-bb39-188762c53aaa" containerID="ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910" exitCode=0
Jul 07 00:28:25.502070 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.501665    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"
Jul 07 00:28:25.502070 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.501692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m" event={"ID":"262a45bd-5a7a-44d5-bb39-188762c53aaa","Type":"ContainerDied","Data":"ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910"}
Jul 07 00:28:25.502070 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.501740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m" event={"ID":"262a45bd-5a7a-44d5-bb39-188762c53aaa","Type":"ContainerDied","Data":"a0dddf43c4a9ff6b8eac9857d26ebc43821feb7e895298c199ed2e6353c03e43"}
Jul 07 00:28:25.502070 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.501762    2564 scope.go:117] "RemoveContainer" containerID="ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910"
Jul 07 00:28:25.511077 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.511064    2564 scope.go:117] "RemoveContainer" containerID="ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910"
Jul 07 00:28:25.511346 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:28:25.511331    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910\": container with ID starting with ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910 not found: ID does not exist" containerID="ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910"
Jul 07 00:28:25.511390 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.511355    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910"} err="failed to get container status \"ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910\": rpc error: code = NotFound desc = could not find container \"ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910\": container with ID starting with ec5a3283eff49daa3312438bfeb0922d7705071072bd45801ad885e281443910 not found: ID does not exist"
Jul 07 00:28:25.523717 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.523672    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"]
Jul 07 00:28:25.525708 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:25.525690    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-7eba6c-849f47f9bd-8rv4m"]
Jul 07 00:28:26.115134 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:28:26.115101    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="262a45bd-5a7a-44d5-bb39-188762c53aaa" path="/var/lib/kubelet/pods/262a45bd-5a7a-44d5-bb39-188762c53aaa/volumes"
Jul 07 00:29:06.448489 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:06.448383    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"]
Jul 07 00:29:06.449015 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:06.448671    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" containerID="cri-o://047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7" gracePeriod=30
Jul 07 00:29:08.680505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:08.680469    2564 patch_prober.go:28] interesting pod/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:29:08.680876 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:08.680524    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:29:11.713362 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.713300    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:29:11.745712 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.745681    2564 generic.go:356] "Generic (PLEG): container finished" podID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerID="047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7" exitCode=0
Jul 07 00:29:11.745879 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.745724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" event={"ID":"fed1130b-cd51-460b-a5b7-d5f4788565da","Type":"ContainerDied","Data":"047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7"}
Jul 07 00:29:11.745879 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.745739    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"
Jul 07 00:29:11.745879 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.745766    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp" event={"ID":"fed1130b-cd51-460b-a5b7-d5f4788565da","Type":"ContainerDied","Data":"1cde4f3ccbd75db0a6e858f9535cd796d8975b0bdad07d50886899b4035cab4f"}
Jul 07 00:29:11.745879 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.745787    2564 scope.go:117] "RemoveContainer" containerID="047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7"
Jul 07 00:29:11.755154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.755134    2564 scope.go:117] "RemoveContainer" containerID="2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"
Jul 07 00:29:11.763214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.763185    2564 scope.go:117] "RemoveContainer" containerID="047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7"
Jul 07 00:29:11.763490 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:29:11.763471    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7\": container with ID starting with 047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7 not found: ID does not exist" containerID="047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7"
Jul 07 00:29:11.763554 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.763501    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7"} err="failed to get container status \"047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7\": rpc error: code = NotFound desc = could not find container \"047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7\": container with ID starting with 047b242ea33076d5dbb87a150c932095f0637db7af2ffb0f048d2eec8d672bb7 not found: ID does not exist"
Jul 07 00:29:11.763554 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.763519    2564 scope.go:117] "RemoveContainer" containerID="2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"
Jul 07 00:29:11.763749 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:29:11.763734    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867\": container with ID starting with 2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867 not found: ID does not exist" containerID="2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"
Jul 07 00:29:11.763798 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.763753    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867"} err="failed to get container status \"2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867\": rpc error: code = NotFound desc = could not find container \"2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867\": container with ID starting with 2d288274f8e723bca2872d898ebe25affc45a19ad5a8f20692ee36a9701ca867 not found: ID does not exist"
Jul 07 00:29:11.853059 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853023    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-socket\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853059 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853067    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-podinfo\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853112    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed1130b-cd51-460b-a5b7-d5f4788565da-istiod-ca-cert\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853129    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-credential-socket\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853153    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-certs\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853179    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-data\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853311 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853274    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2flmr\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-kube-api-access-2flmr\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853589 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853368    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-envoy\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853589 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853373    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:11.853589 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853405    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-token\") pod \"fed1130b-cd51-460b-a5b7-d5f4788565da\" (UID: \"fed1130b-cd51-460b-a5b7-d5f4788565da\") "
Jul 07 00:29:11.853589 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853490    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:11.853589 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853506    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:11.853835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853586    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed1130b-cd51-460b-a5b7-d5f4788565da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:29:11.853835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853802    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed1130b-cd51-460b-a5b7-d5f4788565da-istiod-ca-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.853835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853819    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-credential-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.853835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853829    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.853835 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.853837    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-workload-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.854094 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.854069    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-data" (OuterVolumeSpecName: "istio-data") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:11.855650 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.855618    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:29:11.855755 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.855646    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-kube-api-access-2flmr" (OuterVolumeSpecName: "kube-api-access-2flmr") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "kube-api-access-2flmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:11.855755 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.855626    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-token" (OuterVolumeSpecName: "istio-token") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:11.855755 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.855660    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fed1130b-cd51-460b-a5b7-d5f4788565da" (UID: "fed1130b-cd51-460b-a5b7-d5f4788565da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:11.954592 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.954556    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-data\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.954592 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.954582    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2flmr\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-kube-api-access-2flmr\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.954799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.954604    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-envoy\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.954799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.954613    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:11.954799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:11.954623    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed1130b-cd51-460b-a5b7-d5f4788565da-istio-podinfo\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:12.068297 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:12.068265    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"]
Jul 07 00:29:12.073288 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:12.073256    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7eba6c-openshift-default-665599fdcc-qv7dp"]
Jul 07 00:29:12.120806 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:12.120761    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" path="/var/lib/kubelet/pods/fed1130b-cd51-460b-a5b7-d5f4788565da/volumes"
Jul 07 00:29:14.298122 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298089    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"]
Jul 07 00:29:14.298728 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298708    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262a45bd-5a7a-44d5-bb39-188762c53aaa" containerName="maas-api"
Jul 07 00:29:14.298728 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298730    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="262a45bd-5a7a-44d5-bb39-188762c53aaa" containerName="maas-api"
Jul 07 00:29:14.298870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298743    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy"
Jul 07 00:29:14.298870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298751    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy"
Jul 07 00:29:14.298971 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298882    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy"
Jul 07 00:29:14.298971 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298894    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy"
Jul 07 00:29:14.298971 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.298908    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="262a45bd-5a7a-44d5-bb39-188762c53aaa" containerName="maas-api"
Jul 07 00:29:14.303592 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.303565    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.306750 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.306730    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2bebf4-openshift-default-dockercfg-nbf2b\""
Jul 07 00:29:14.311181 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.311158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"]
Jul 07 00:29:14.474906 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.474871    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14028f5c-af56-430b-b284-cba7954ad21d-istiod-ca-cert\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475097 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.474915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-credential-socket\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475097 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.474948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14028f5c-af56-430b-b284-cba7954ad21d-istio-podinfo\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475097 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.475015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbpxd\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-kube-api-access-qbpxd\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475097 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.475055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-socket\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475269 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.475140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-certs\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475269 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.475170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-istio-token\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475269 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.475214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-data\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.475385 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.475287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-envoy\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-istio-token\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-data\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-envoy\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576609    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14028f5c-af56-430b-b284-cba7954ad21d-istiod-ca-cert\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-credential-socket\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14028f5c-af56-430b-b284-cba7954ad21d-istio-podinfo\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbpxd\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-kube-api-access-qbpxd\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.576799 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-socket\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.577056 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-certs\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.577056 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.576893    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-data\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.577159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.577048    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-credential-socket\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.577159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.577115    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-socket\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.577276 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.577175    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-certs\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.577276 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.577263    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14028f5c-af56-430b-b284-cba7954ad21d-istiod-ca-cert\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.578901 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.578882    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-envoy\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.579202 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.579184    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14028f5c-af56-430b-b284-cba7954ad21d-istio-podinfo\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.584591 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.584563    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-istio-token\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.584591 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.584576    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbpxd\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-kube-api-access-qbpxd\") pod \"e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") " pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.614669 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.614631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:14.758877 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.758851    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"]
Jul 07 00:29:14.759408 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:29:14.759382    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14028f5c_af56_430b_b284_cba7954ad21d.slice/crio-24dbb8a9ae6310f3c2e853319eca6568c7e249401c4bf617d4844db224e91efe WatchSource:0}: Error finding container 24dbb8a9ae6310f3c2e853319eca6568c7e249401c4bf617d4844db224e91efe: Status 404 returned error can't find the container with id 24dbb8a9ae6310f3c2e853319eca6568c7e249401c4bf617d4844db224e91efe
Jul 07 00:29:14.761653 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.761616    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:14.761753 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.761698    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:14.761753 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.761745    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:14.767025 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:14.767000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" event={"ID":"14028f5c-af56-430b-b284-cba7954ad21d","Type":"ContainerStarted","Data":"24dbb8a9ae6310f3c2e853319eca6568c7e249401c4bf617d4844db224e91efe"}
Jul 07 00:29:15.772911 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:15.772869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" event={"ID":"14028f5c-af56-430b-b284-cba7954ad21d","Type":"ContainerStarted","Data":"3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"}
Jul 07 00:29:15.796417 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:15.796367    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podStartSLOduration=1.796350587 podStartE2EDuration="1.796350587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:15.794533521 +0000 UTC m=+1988.207547447" watchObservedRunningTime="2026-07-07 00:29:15.796350587 +0000 UTC m=+1988.209364495"
Jul 07 00:29:16.615069 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:16.615033    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:16.620240 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:16.620213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:16.777739 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:16.777699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:16.778734 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:16.778713    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:32.819654 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:32.819605    2564 patch_prober.go:28] interesting pod/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 00:29:32.820038 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:32.819677    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:29:34.879936 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:34.879900    2564 generic.go:356] "Generic (PLEG): container finished" podID="14028f5c-af56-430b-b284-cba7954ad21d" containerID="3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936" exitCode=0
Jul 07 00:29:34.880435 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:34.879975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" event={"ID":"14028f5c-af56-430b-b284-cba7954ad21d","Type":"ContainerDied","Data":"3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"}
Jul 07 00:29:34.880435 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:34.880361    2564 scope.go:117] "RemoveContainer" containerID="3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"
Jul 07 00:29:34.881352 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:34.881327    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:34.881412 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:34.881397    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:34.881472 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:34.881435    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:35.887233 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:35.887200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" event={"ID":"14028f5c-af56-430b-b284-cba7954ad21d","Type":"ContainerStarted","Data":"b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d"}
Jul 07 00:29:36.615044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:36.615003    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:36.615293 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:36.615258    2564 patch_prober.go:28] interesting pod/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 07 00:29:36.615370 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:36.615316    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 07 00:29:37.615245 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:37.615199    2564 patch_prober.go:28] interesting pod/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 07 00:29:37.615736 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:37.615271    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 07 00:29:38.615353 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:38.615313    2564 patch_prober.go:28] interesting pod/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 07 00:29:38.615804 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:38.615372    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 07 00:29:39.615819 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:39.615772    2564 patch_prober.go:28] interesting pod/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 07 00:29:39.616217 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:39.615847    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 07 00:29:40.618935 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:40.618902    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:40.619356 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:40.619178    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:40.619832 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:40.619817    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:46.159211 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:46.159170    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"]
Jul 07 00:29:46.159669 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:46.159370    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy" containerID="cri-o://b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d" gracePeriod=30
Jul 07 00:29:51.409400 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.409373    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:51.545479 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545350    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbpxd\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-kube-api-access-qbpxd\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545479 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545395    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-certs\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545497    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-socket\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14028f5c-af56-430b-b284-cba7954ad21d-istio-podinfo\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545594    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-credential-socket\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545621    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-istio-token\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14028f5c-af56-430b-b284-cba7954ad21d-istiod-ca-cert\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545679    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-envoy\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545714    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:51.545733 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545736    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-data\") pod \"14028f5c-af56-430b-b284-cba7954ad21d\" (UID: \"14028f5c-af56-430b-b284-cba7954ad21d\") "
Jul 07 00:29:51.546154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545844    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:51.546154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.545941    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:51.546154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.546104    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.546154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.546122    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-workload-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.546154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.546135    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-credential-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.546154 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.546137    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14028f5c-af56-430b-b284-cba7954ad21d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:29:51.546562 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.546544    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-data" (OuterVolumeSpecName: "istio-data") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:51.547794 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.547773    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-kube-api-access-qbpxd" (OuterVolumeSpecName: "kube-api-access-qbpxd") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "kube-api-access-qbpxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:51.547989 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.547964    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14028f5c-af56-430b-b284-cba7954ad21d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:29:51.548200 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.548186    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:29:51.548303 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.548289    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-istio-token" (OuterVolumeSpecName: "istio-token") pod "14028f5c-af56-430b-b284-cba7954ad21d" (UID: "14028f5c-af56-430b-b284-cba7954ad21d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:29:51.647576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.647542    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-data\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.647576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.647571    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbpxd\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-kube-api-access-qbpxd\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.647576 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.647581    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14028f5c-af56-430b-b284-cba7954ad21d-istio-podinfo\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.647828 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.647590    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14028f5c-af56-430b-b284-cba7954ad21d-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.647828 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.647598    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14028f5c-af56-430b-b284-cba7954ad21d-istiod-ca-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.647828 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.647606    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14028f5c-af56-430b-b284-cba7954ad21d-istio-envoy\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:29:51.971012 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.970980    2564 generic.go:356] "Generic (PLEG): container finished" podID="14028f5c-af56-430b-b284-cba7954ad21d" containerID="b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d" exitCode=0
Jul 07 00:29:51.971180 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.971031    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"
Jul 07 00:29:51.971180 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.971058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" event={"ID":"14028f5c-af56-430b-b284-cba7954ad21d","Type":"ContainerDied","Data":"b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d"}
Jul 07 00:29:51.971180 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.971118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5" event={"ID":"14028f5c-af56-430b-b284-cba7954ad21d","Type":"ContainerDied","Data":"24dbb8a9ae6310f3c2e853319eca6568c7e249401c4bf617d4844db224e91efe"}
Jul 07 00:29:51.971180 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.971150    2564 scope.go:117] "RemoveContainer" containerID="b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d"
Jul 07 00:29:51.980742 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.980724    2564 scope.go:117] "RemoveContainer" containerID="3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"
Jul 07 00:29:51.989213 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.989193    2564 scope.go:117] "RemoveContainer" containerID="b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d"
Jul 07 00:29:51.989528 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:29:51.989504    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d\": container with ID starting with b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d not found: ID does not exist" containerID="b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d"
Jul 07 00:29:51.989620 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.989537    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d"} err="failed to get container status \"b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d\": rpc error: code = NotFound desc = could not find container \"b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d\": container with ID starting with b820d88be714fd5a08e270588cf4ea607e70a12cc2b0c7e42cd7c222a7b46c4d not found: ID does not exist"
Jul 07 00:29:51.989620 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.989554    2564 scope.go:117] "RemoveContainer" containerID="3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"
Jul 07 00:29:51.989835 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:29:51.989818    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936\": container with ID starting with 3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936 not found: ID does not exist" containerID="3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"
Jul 07 00:29:51.989884 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.989841    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936"} err="failed to get container status \"3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936\": rpc error: code = NotFound desc = could not find container \"3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936\": container with ID starting with 3f3e5bcd73a38a793732de50b94840b462c2313115a211eb8e05b55615428936 not found: ID does not exist"
Jul 07 00:29:51.996334 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:51.996308    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"]
Jul 07 00:29:52.001977 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:52.001950    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2bebf4-openshift-default-78dbd58fdf-c4bl5"]
Jul 07 00:29:52.115993 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:52.115959    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14028f5c-af56-430b-b284-cba7954ad21d" path="/var/lib/kubelet/pods/14028f5c-af56-430b-b284-cba7954ad21d/volumes"
Jul 07 00:29:54.526193 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526160    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"]
Jul 07 00:29:54.526632 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526618    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy"
Jul 07 00:29:54.526676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526634    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy"
Jul 07 00:29:54.526676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526644    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy"
Jul 07 00:29:54.526676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526651    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy"
Jul 07 00:29:54.526676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526659    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy"
Jul 07 00:29:54.526676 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526666    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed1130b-cd51-460b-a5b7-d5f4788565da" containerName="istio-proxy"
Jul 07 00:29:54.526837 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.526742    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy"
Jul 07 00:29:54.529874 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.529853    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.532856 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.532825    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ea3965-openshift-default-dockercfg-69kxx\""
Jul 07 00:29:54.544921 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.544893    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"]
Jul 07 00:29:54.572310 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-token\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572310 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-credential-socket\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572567 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-envoy\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572567 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572385    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jbp\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-kube-api-access-c7jbp\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572567 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-socket\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572567 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-data\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572567 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-certs\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572567 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istiod-ca-cert\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.572768 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.572611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-podinfo\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673105 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-token\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673105 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-credential-socket\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-envoy\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jbp\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-kube-api-access-c7jbp\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-socket\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673226    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-data\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-certs\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istiod-ca-cert\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673365 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-podinfo\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673751 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673628    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-credential-socket\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673751 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673725    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-socket\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673855 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673762    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-data\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.673855 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.673785    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-certs\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.674056 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.674040    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istiod-ca-cert\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.676111 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.676088    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-envoy\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.676434 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.676412    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-podinfo\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.689129 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.689099    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-token\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.689624 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.689607    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jbp\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-kube-api-access-c7jbp\") pod \"e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") " pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.840216 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.840181    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:54.984245 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.984217    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"]
Jul 07 00:29:54.986050 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:29:54.986009    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf7d58c_fcf7_4522_b132_97e4dcf8d56a.slice/crio-ca64d0e8d355d3b045b831d6cf2616f244d47f577f8e5f705b7892eb3320dbf6 WatchSource:0}: Error finding container ca64d0e8d355d3b045b831d6cf2616f244d47f577f8e5f705b7892eb3320dbf6: Status 404 returned error can't find the container with id ca64d0e8d355d3b045b831d6cf2616f244d47f577f8e5f705b7892eb3320dbf6
Jul 07 00:29:54.988233 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.988199    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:54.988312 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.988275    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:54.988351 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:54.988312    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:29:55.993271 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:55.993236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" event={"ID":"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a","Type":"ContainerStarted","Data":"ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"}
Jul 07 00:29:55.993773 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:55.993290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" event={"ID":"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a","Type":"ContainerStarted","Data":"ca64d0e8d355d3b045b831d6cf2616f244d47f577f8e5f705b7892eb3320dbf6"}
Jul 07 00:29:56.013569 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:56.013502    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podStartSLOduration=2.013481209 podStartE2EDuration="2.013481209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:56.012632915 +0000 UTC m=+2028.425646825" watchObservedRunningTime="2026-07-07 00:29:56.013481209 +0000 UTC m=+2028.426495118"
Jul 07 00:29:56.840717 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:56.840673    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:56.846516 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:56.846482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:56.998795 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:56.998759    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:29:56.999887 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:29:56.999867    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:30:11.085241 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.085153    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerID="ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb" exitCode=0
Jul 07 00:30:11.085241 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.085215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" event={"ID":"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a","Type":"ContainerDied","Data":"ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"}
Jul 07 00:30:11.085764 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.085617    2564 scope.go:117] "RemoveContainer" containerID="ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"
Jul 07 00:30:11.086666 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.086637    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:30:11.086730 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.086710    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:30:11.086769 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.086743    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 07 00:30:11.998984 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:11.998949    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:30:12.092302 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:12.092264    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" event={"ID":"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a","Type":"ContainerStarted","Data":"786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a"}
Jul 07 00:30:12.840716 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:12.840672    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:30:12.842348 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:12.842309    2564 patch_prober.go:28] interesting pod/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 07 00:30:12.842646 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:12.842611    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 07 00:30:13.841199 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:13.841152    2564 patch_prober.go:28] interesting pod/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 07 00:30:13.841860 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:13.841822    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 07 00:30:14.841240 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:14.841202    2564 patch_prober.go:28] interesting pod/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 07 00:30:14.841834 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:14.841803    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 07 00:30:15.840938 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:15.840898    2564 patch_prober.go:28] interesting pod/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Jul 07 00:30:15.841238 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:15.841208    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Jul 07 00:30:16.844730 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:16.844702    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:30:16.845123 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:16.844981    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:30:16.845653 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:30:16.845640    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:31:08.210069 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:31:08.210040    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:31:08.211605 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:31:08.211585    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:36:08.242980 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:08.242947    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:36:08.244768 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:08.244743    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:36:14.394301 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.394265    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911069003312"]
Jul 07 00:36:14.394757 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.394738    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="14028f5c-af56-430b-b284-cba7954ad21d" containerName="istio-proxy"
Jul 07 00:36:14.397891 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.397872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:14.403869 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.403842    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911069003312"]
Jul 07 00:36:14.519920 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.519840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcxrd\" (UniqueName: \"kubernetes.io/projected/238f960e-257a-485d-a01a-46e1dfdfbea7-kube-api-access-pcxrd\") pod \"test-curl-7404-139911069003312\" (UID: \"238f960e-257a-485d-a01a-46e1dfdfbea7\") " pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:14.621419 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.621382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcxrd\" (UniqueName: \"kubernetes.io/projected/238f960e-257a-485d-a01a-46e1dfdfbea7-kube-api-access-pcxrd\") pod \"test-curl-7404-139911069003312\" (UID: \"238f960e-257a-485d-a01a-46e1dfdfbea7\") " pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:14.630491 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.630436    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcxrd\" (UniqueName: \"kubernetes.io/projected/238f960e-257a-485d-a01a-46e1dfdfbea7-kube-api-access-pcxrd\") pod \"test-curl-7404-139911069003312\" (UID: \"238f960e-257a-485d-a01a-46e1dfdfbea7\") " pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:14.707915 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.707884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:14.848212 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.848183    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911069003312"]
Jul 07 00:36:14.850841 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:36:14.850806    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod238f960e_257a_485d_a01a_46e1dfdfbea7.slice/crio-2799990a987c32cb549d0270a008a043f2f5263b602b253f333253f4e22c7750 WatchSource:0}: Error finding container 2799990a987c32cb549d0270a008a043f2f5263b602b253f333253f4e22c7750: Status 404 returned error can't find the container with id 2799990a987c32cb549d0270a008a043f2f5263b602b253f333253f4e22c7750
Jul 07 00:36:14.852191 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.852170    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:36:14.891124 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:14.891092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911069003312" event={"ID":"238f960e-257a-485d-a01a-46e1dfdfbea7","Type":"ContainerStarted","Data":"2799990a987c32cb549d0270a008a043f2f5263b602b253f333253f4e22c7750"}
Jul 07 00:36:15.897347 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:15.897251    2564 generic.go:356] "Generic (PLEG): container finished" podID="238f960e-257a-485d-a01a-46e1dfdfbea7" containerID="dfb8016c627fe7a1ff9e9f7eedfef81d7435dc8bd1cf424764ea7128514d58ed" exitCode=0
Jul 07 00:36:15.897347 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:15.897317    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911069003312" event={"ID":"238f960e-257a-485d-a01a-46e1dfdfbea7","Type":"ContainerDied","Data":"dfb8016c627fe7a1ff9e9f7eedfef81d7435dc8bd1cf424764ea7128514d58ed"}
Jul 07 00:36:17.041924 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.041902    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:17.079309 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.079273    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911069003312_238f960e-257a-485d-a01a-46e1dfdfbea7/test-curl-7404-139911069003312/0.log"
Jul 07 00:36:17.104842 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.104809    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911069003312"]
Jul 07 00:36:17.111057 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.111034    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911069003312"]
Jul 07 00:36:17.146621 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.146578    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcxrd\" (UniqueName: \"kubernetes.io/projected/238f960e-257a-485d-a01a-46e1dfdfbea7-kube-api-access-pcxrd\") pod \"238f960e-257a-485d-a01a-46e1dfdfbea7\" (UID: \"238f960e-257a-485d-a01a-46e1dfdfbea7\") "
Jul 07 00:36:17.148627 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.148598    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/238f960e-257a-485d-a01a-46e1dfdfbea7-kube-api-access-pcxrd" (OuterVolumeSpecName: "kube-api-access-pcxrd") pod "238f960e-257a-485d-a01a-46e1dfdfbea7" (UID: "238f960e-257a-485d-a01a-46e1dfdfbea7"). InnerVolumeSpecName "kube-api-access-pcxrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:17.239812 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.239781    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911069001632"]
Jul 07 00:36:17.240220 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.240207    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238f960e-257a-485d-a01a-46e1dfdfbea7" containerName="test-curl-7404-139911069003312"
Jul 07 00:36:17.240271 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.240223    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="238f960e-257a-485d-a01a-46e1dfdfbea7" containerName="test-curl-7404-139911069003312"
Jul 07 00:36:17.241444 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.240879    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="238f960e-257a-485d-a01a-46e1dfdfbea7" containerName="test-curl-7404-139911069003312"
Jul 07 00:36:17.245478 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.245194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:17.248932 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.248243    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcxrd\" (UniqueName: \"kubernetes.io/projected/238f960e-257a-485d-a01a-46e1dfdfbea7-kube-api-access-pcxrd\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:36:17.251886 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.251860    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911069001632"]
Jul 07 00:36:17.348931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.348897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnhq\" (UniqueName: \"kubernetes.io/projected/9bbb2b59-5cf0-4fe3-b114-53ca033cba28-kube-api-access-dgnhq\") pod \"test-curl-7404-139911069001632\" (UID: \"9bbb2b59-5cf0-4fe3-b114-53ca033cba28\") " pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:17.449821 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.449713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnhq\" (UniqueName: \"kubernetes.io/projected/9bbb2b59-5cf0-4fe3-b114-53ca033cba28-kube-api-access-dgnhq\") pod \"test-curl-7404-139911069001632\" (UID: \"9bbb2b59-5cf0-4fe3-b114-53ca033cba28\") " pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:17.458494 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.458439    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnhq\" (UniqueName: \"kubernetes.io/projected/9bbb2b59-5cf0-4fe3-b114-53ca033cba28-kube-api-access-dgnhq\") pod \"test-curl-7404-139911069001632\" (UID: \"9bbb2b59-5cf0-4fe3-b114-53ca033cba28\") " pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:17.557647 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.557602    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:17.695301 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.695270    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911069001632"]
Jul 07 00:36:17.696118 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:36:17.696088    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bbb2b59_5cf0_4fe3_b114_53ca033cba28.slice/crio-49b92f895c8418cf8086b9a1ed8b31ad89ae9980ebca9e4daef61a8f489ed9ad WatchSource:0}: Error finding container 49b92f895c8418cf8086b9a1ed8b31ad89ae9980ebca9e4daef61a8f489ed9ad: Status 404 returned error can't find the container with id 49b92f895c8418cf8086b9a1ed8b31ad89ae9980ebca9e4daef61a8f489ed9ad
Jul 07 00:36:17.908717 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.908682    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:17.908717 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.908710    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2799990a987c32cb549d0270a008a043f2f5263b602b253f333253f4e22c7750"
Jul 07 00:36:17.910003 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:17.909968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911069001632" event={"ID":"9bbb2b59-5cf0-4fe3-b114-53ca033cba28","Type":"ContainerStarted","Data":"49b92f895c8418cf8086b9a1ed8b31ad89ae9980ebca9e4daef61a8f489ed9ad"}
Jul 07 00:36:17.920147 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:36:17.920120    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911069003312\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="238f960e-257a-485d-a01a-46e1dfdfbea7" pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:18.113334 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:36:18.113303    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911069003312\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="238f960e-257a-485d-a01a-46e1dfdfbea7" pod="opendatahub/test-curl-7404-139911069003312"
Jul 07 00:36:18.114841 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:18.114821    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="238f960e-257a-485d-a01a-46e1dfdfbea7" path="/var/lib/kubelet/pods/238f960e-257a-485d-a01a-46e1dfdfbea7/volumes"
Jul 07 00:36:18.915905 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:18.915874    2564 generic.go:356] "Generic (PLEG): container finished" podID="9bbb2b59-5cf0-4fe3-b114-53ca033cba28" containerID="553d16285a5cff2ed35aed556328df87f6b899bfe007ffb6696bced52e301097" exitCode=0
Jul 07 00:36:18.916089 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:18.915926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911069001632" event={"ID":"9bbb2b59-5cf0-4fe3-b114-53ca033cba28","Type":"ContainerDied","Data":"553d16285a5cff2ed35aed556328df87f6b899bfe007ffb6696bced52e301097"}
Jul 07 00:36:20.061307 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.061283    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:20.080341 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.080310    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911069001632_9bbb2b59-5cf0-4fe3-b114-53ca033cba28/test-curl-7404-139911069001632/0.log"
Jul 07 00:36:20.102091 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.102050    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911069001632"]
Jul 07 00:36:20.105318 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.105292    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911069001632"]
Jul 07 00:36:20.176931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.176847    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgnhq\" (UniqueName: \"kubernetes.io/projected/9bbb2b59-5cf0-4fe3-b114-53ca033cba28-kube-api-access-dgnhq\") pod \"9bbb2b59-5cf0-4fe3-b114-53ca033cba28\" (UID: \"9bbb2b59-5cf0-4fe3-b114-53ca033cba28\") "
Jul 07 00:36:20.178934 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.178908    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bbb2b59-5cf0-4fe3-b114-53ca033cba28-kube-api-access-dgnhq" (OuterVolumeSpecName: "kube-api-access-dgnhq") pod "9bbb2b59-5cf0-4fe3-b114-53ca033cba28" (UID: "9bbb2b59-5cf0-4fe3-b114-53ca033cba28"). InnerVolumeSpecName "kube-api-access-dgnhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:20.278411 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.278381    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgnhq\" (UniqueName: \"kubernetes.io/projected/9bbb2b59-5cf0-4fe3-b114-53ca033cba28-kube-api-access-dgnhq\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:36:20.336705 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.336670    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7404-139911071281632"]
Jul 07 00:36:20.337116 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.337100    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb2b59-5cf0-4fe3-b114-53ca033cba28" containerName="test-curl-7404-139911069001632"
Jul 07 00:36:20.337158 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.337119    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb2b59-5cf0-4fe3-b114-53ca033cba28" containerName="test-curl-7404-139911069001632"
Jul 07 00:36:20.337207 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.337198    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bbb2b59-5cf0-4fe3-b114-53ca033cba28" containerName="test-curl-7404-139911069001632"
Jul 07 00:36:20.340321 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.340306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:20.346048 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.346018    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911071281632"]
Jul 07 00:36:20.481131 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.481040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q798\" (UniqueName: \"kubernetes.io/projected/2bdcbb93-4747-4452-adab-e782a7244437-kube-api-access-2q798\") pod \"test-curl-7404-139911071281632\" (UID: \"2bdcbb93-4747-4452-adab-e782a7244437\") " pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:20.582656 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.582610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q798\" (UniqueName: \"kubernetes.io/projected/2bdcbb93-4747-4452-adab-e782a7244437-kube-api-access-2q798\") pod \"test-curl-7404-139911071281632\" (UID: \"2bdcbb93-4747-4452-adab-e782a7244437\") " pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:20.591616 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.591594    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q798\" (UniqueName: \"kubernetes.io/projected/2bdcbb93-4747-4452-adab-e782a7244437-kube-api-access-2q798\") pod \"test-curl-7404-139911071281632\" (UID: \"2bdcbb93-4747-4452-adab-e782a7244437\") " pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:20.650343 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.650305    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:20.784601 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.784466    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7404-139911071281632"]
Jul 07 00:36:20.786813 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:36:20.786778    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bdcbb93_4747_4452_adab_e782a7244437.slice/crio-33ece94331277e0bf9ae9a096574712068475e2ee4cf66dee90e48b10f5007d8 WatchSource:0}: Error finding container 33ece94331277e0bf9ae9a096574712068475e2ee4cf66dee90e48b10f5007d8: Status 404 returned error can't find the container with id 33ece94331277e0bf9ae9a096574712068475e2ee4cf66dee90e48b10f5007d8
Jul 07 00:36:20.928249 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.928223    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911069001632"
Jul 07 00:36:20.928441 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.928286    2564 scope.go:117] "RemoveContainer" containerID="553d16285a5cff2ed35aed556328df87f6b899bfe007ffb6696bced52e301097"
Jul 07 00:36:20.929772 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:20.929713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911071281632" event={"ID":"2bdcbb93-4747-4452-adab-e782a7244437","Type":"ContainerStarted","Data":"33ece94331277e0bf9ae9a096574712068475e2ee4cf66dee90e48b10f5007d8"}
Jul 07 00:36:21.936427 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:21.936388    2564 generic.go:356] "Generic (PLEG): container finished" podID="2bdcbb93-4747-4452-adab-e782a7244437" containerID="c133cfd0d401a4080d3acb2d911e83e54d9dec097f427295ee890cf631a74e2f" exitCode=0
Jul 07 00:36:21.936876 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:21.936488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7404-139911071281632" event={"ID":"2bdcbb93-4747-4452-adab-e782a7244437","Type":"ContainerDied","Data":"c133cfd0d401a4080d3acb2d911e83e54d9dec097f427295ee890cf631a74e2f"}
Jul 07 00:36:22.114859 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:22.114826    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bbb2b59-5cf0-4fe3-b114-53ca033cba28" path="/var/lib/kubelet/pods/9bbb2b59-5cf0-4fe3-b114-53ca033cba28/volumes"
Jul 07 00:36:23.081445 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.081423    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:23.101059 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.101036    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7404-139911071281632_2bdcbb93-4747-4452-adab-e782a7244437/test-curl-7404-139911071281632/0.log"
Jul 07 00:36:23.123928 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.123895    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7404-139911071281632"]
Jul 07 00:36:23.128303 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.128275    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7404-139911071281632"]
Jul 07 00:36:23.209581 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.209494    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q798\" (UniqueName: \"kubernetes.io/projected/2bdcbb93-4747-4452-adab-e782a7244437-kube-api-access-2q798\") pod \"2bdcbb93-4747-4452-adab-e782a7244437\" (UID: \"2bdcbb93-4747-4452-adab-e782a7244437\") "
Jul 07 00:36:23.211665 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.211645    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bdcbb93-4747-4452-adab-e782a7244437-kube-api-access-2q798" (OuterVolumeSpecName: "kube-api-access-2q798") pod "2bdcbb93-4747-4452-adab-e782a7244437" (UID: "2bdcbb93-4747-4452-adab-e782a7244437"). InnerVolumeSpecName "kube-api-access-2q798". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:23.311149 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.311115    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q798\" (UniqueName: \"kubernetes.io/projected/2bdcbb93-4747-4452-adab-e782a7244437-kube-api-access-2q798\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:36:23.948044 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.948015    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:23.948216 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:23.948022    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33ece94331277e0bf9ae9a096574712068475e2ee4cf66dee90e48b10f5007d8"
Jul 07 00:36:23.958469 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:36:23.958431    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7404-139911071281632\" is forbidden: User \"system:node:ip-10-0-141-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-123.ec2.internal' and this object" podUID="2bdcbb93-4747-4452-adab-e782a7244437" pod="opendatahub/test-curl-7404-139911071281632"
Jul 07 00:36:24.116064 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:24.116023    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bdcbb93-4747-4452-adab-e782a7244437" path="/var/lib/kubelet/pods/2bdcbb93-4747-4452-adab-e782a7244437/volumes"
Jul 07 00:36:57.623941 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:57.623862    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"]
Jul 07 00:36:57.624439 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:36:57.624129    2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy" containerID="cri-o://786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a" gracePeriod=30
Jul 07 00:37:01.847544 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:01.847507    2564 patch_prober.go:28] interesting pod/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:37:01.847930 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:01.847578    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:37:02.890094 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.890031    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:37:02.965990 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.965958    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-podinfo\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966011    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7jbp\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-kube-api-access-c7jbp\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966095    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-socket\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966147    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-data\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966179    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-envoy\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istiod-ca-cert\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966235    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-certs\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966259    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-credential-socket\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966280 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966284    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-token\") pod \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\" (UID: \"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a\") "
Jul 07 00:37:02.966770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966306    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:37:02.966770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966513    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:37:02.966770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966616    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:37:02.966770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966681    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:37:02.966770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966746    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:02.966770 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966764    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-workload-certs\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:02.967037 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.966778    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-credential-socket\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:02.967037 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.967021    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:37:02.968635 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.968608    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-kube-api-access-c7jbp" (OuterVolumeSpecName: "kube-api-access-c7jbp") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "kube-api-access-c7jbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:37:02.968635 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.968617    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:37:02.968805 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.968698    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:37:02.968805 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:02.968738    2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" (UID: "fcf7d58c-fcf7-4522-b132-97e4dcf8d56a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:37:03.067686 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.067647    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-podinfo\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:03.067686 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.067679    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7jbp\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-kube-api-access-c7jbp\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:03.067686 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.067691    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-data\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:03.067951 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.067700    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-envoy\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:03.067951 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.067709    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istiod-ca-cert\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:03.067951 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.067717    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a-istio-token\") on node \"ip-10-0-141-123.ec2.internal\" DevicePath \"\""
Jul 07 00:37:03.153789 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.153694    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerID="786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a" exitCode=0
Jul 07 00:37:03.153789 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.153755    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"
Jul 07 00:37:03.153789 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.153766    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" event={"ID":"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a","Type":"ContainerDied","Data":"786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a"}
Jul 07 00:37:03.154022 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.153809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb" event={"ID":"fcf7d58c-fcf7-4522-b132-97e4dcf8d56a","Type":"ContainerDied","Data":"ca64d0e8d355d3b045b831d6cf2616f244d47f577f8e5f705b7892eb3320dbf6"}
Jul 07 00:37:03.154022 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.153832    2564 scope.go:117] "RemoveContainer" containerID="786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a"
Jul 07 00:37:03.163680 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.163659    2564 scope.go:117] "RemoveContainer" containerID="ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"
Jul 07 00:37:03.172274 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.172253    2564 scope.go:117] "RemoveContainer" containerID="786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a"
Jul 07 00:37:03.172546 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:37:03.172528    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a\": container with ID starting with 786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a not found: ID does not exist" containerID="786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a"
Jul 07 00:37:03.172597 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.172555    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a"} err="failed to get container status \"786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a\": rpc error: code = NotFound desc = could not find container \"786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a\": container with ID starting with 786a72112bb10067d5dc0456ee6e34b21beabd508a931ecf9a1964858ee1629a not found: ID does not exist"
Jul 07 00:37:03.172597 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.172574    2564 scope.go:117] "RemoveContainer" containerID="ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"
Jul 07 00:37:03.172814 ip-10-0-141-123 kubenswrapper[2564]: E0707 00:37:03.172795    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb\": container with ID starting with ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb not found: ID does not exist" containerID="ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"
Jul 07 00:37:03.172875 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.172823    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb"} err="failed to get container status \"ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb\": rpc error: code = NotFound desc = could not find container \"ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb\": container with ID starting with ae97ed91542d53b622d6fe510c0a129fd8a7a6e9ad5696113ff696725c5236bb not found: ID does not exist"
Jul 07 00:37:03.177449 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.177427    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"]
Jul 07 00:37:03.181214 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:03.181187    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ea3965-openshift-default-65bc8fdf5f-5jfjb"]
Jul 07 00:37:04.115420 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:04.115389    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" path="/var/lib/kubelet/pods/fcf7d58c-fcf7-4522-b132-97e4dcf8d56a/volumes"
Jul 07 00:37:05.310298 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:05.310267    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c8db5d754-stvkt_afec69ca-692e-4ae8-8b06-5f7cc001bbe9/authorino/0.log"
Jul 07 00:37:10.019066 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:10.019025    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rb8zq_a7d5d300-2afb-4e36-b6b7-62846e3aa0fc/manager/0.log"
Jul 07 00:37:10.158543 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:10.158510    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fpsgs_b3cd4eaf-58f0-4620-931f-6631a3d2d229/manager/0.log"
Jul 07 00:37:10.285480 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:10.285389    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-9bd647d7b-l86f6_5eb1b25c-e46d-4f2c-9332-38a6c55e5a36/maas-api/0.log"
Jul 07 00:37:10.658351 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:10.658313    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6966b9487-qhhx6_b4ecc55e-30a9-4b5f-b853-e29cb7ef6785/manager/0.log"
Jul 07 00:37:10.779737 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:10.779707    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-tsrs9_6c6914c8-2db9-46ad-b6d5-788ec2769ecc/server/0.log"
Jul 07 00:37:10.923923 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:10.923848    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-f7xn7_0250feb6-d7bc-429e-bde9-d16d3e95ed96/manager/0.log"
Jul 07 00:37:11.176513 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:11.176421    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-w8xms_fa93f0f7-0145-4ac0-9309-d9fb429247d1/copy-manifests/0.log"
Jul 07 00:37:11.183160 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:11.183132    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-w8xms_fa93f0f7-0145-4ac0-9309-d9fb429247d1/manager/0.log"
Jul 07 00:37:11.418992 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:11.418961    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mm8qm_ac916e15-2868-41ec-92b6-9af3ce0d710d/postgres/0.log"
Jul 07 00:37:12.751552 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:12.751523    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c8db5d754-stvkt_afec69ca-692e-4ae8-8b06-5f7cc001bbe9/authorino/0.log"
Jul 07 00:37:12.880269 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:12.880235    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nxjhp_e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4/manager/0.log"
Jul 07 00:37:13.225317 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:13.225273    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p65st_c3c0e3f7-d421-4c6c-ba89-117db18e2778/registry-server/0.log"
Jul 07 00:37:13.633930 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:13.633894    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lfx9p_e64af115-de23-4f39-9bc6-eb09bdb125ca/manager/0.log"
Jul 07 00:37:13.994058 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:13.993972    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678w6qh2_6609b6e5-bb86-4870-ae30-f1d6a11a661d/istio-proxy/0.log"
Jul 07 00:37:14.365035 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:14.364985    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-x88l5_cdb0be87-65eb-40b6-9c56-e07ddee42b88/kube-auth-proxy/0.log"
Jul 07 00:37:14.502519 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:14.502488    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/1.log"
Jul 07 00:37:14.752943 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:14.752859    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kd7mx_ea4d2988-31b5-4bb6-b71f-77554ff8c10e/payload-processing/0.log"
Jul 07 00:37:14.864932 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:14.864901    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68846f8cb5-nx52d_e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726/router/0.log"
Jul 07 00:37:15.232147 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:15.232115    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9ct5n_b3ef597d-b888-4ddb-8c64-31316a83bfd1/main/0.log"
Jul 07 00:37:15.712263 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:15.712230    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-k2fn6_6b451d6d-5945-4a7d-b4bd-a377b90fad7b/main/0.log"
Jul 07 00:37:15.843814 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:15.843775    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-5fdwx_fcf724a0-dfe2-4d5f-adb2-fd7d0ed70555/main/0.log"
Jul 07 00:37:23.085862 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:23.085828    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5rgcf_e044b7a4-d4c4-444c-a225-ffc0f2b5eb15/global-pull-secret-syncer/0.log"
Jul 07 00:37:23.259423 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:23.259392    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qvk4s_9930d01b-c2fb-45eb-950c-c36e2bda9296/konnectivity-agent/0.log"
Jul 07 00:37:23.368705 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:23.368667    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-123.ec2.internal_bc8b346ff26bd2da637db45021bfde3d/haproxy/0.log"
Jul 07 00:37:28.796260 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:28.796227    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c8db5d754-stvkt_afec69ca-692e-4ae8-8b06-5f7cc001bbe9/authorino/0.log"
Jul 07 00:37:28.854264 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:28.854235    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nxjhp_e0b3a7ef-6c9c-469d-a91d-65fa2a049fb4/manager/0.log"
Jul 07 00:37:28.957947 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:28.957914    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p65st_c3c0e3f7-d421-4c6c-ba89-117db18e2778/registry-server/0.log"
Jul 07 00:37:29.245341 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:29.245306    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lfx9p_e64af115-de23-4f39-9bc6-eb09bdb125ca/manager/0.log"
Jul 07 00:37:31.132284 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:31.132253    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-djml9_061a0d8e-ce66-4d25-b524-839070cbd53c/cluster-monitoring-operator/0.log"
Jul 07 00:37:31.301331 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:31.301286    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g7j7q_37c86b59-d35e-4a6c-89c0-d29199c02372/node-exporter/0.log"
Jul 07 00:37:31.322701 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:31.322652    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g7j7q_37c86b59-d35e-4a6c-89c0-d29199c02372/kube-rbac-proxy/0.log"
Jul 07 00:37:31.354976 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:31.354949    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g7j7q_37c86b59-d35e-4a6c-89c0-d29199c02372/init-textfile/0.log"
Jul 07 00:37:33.258786 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.258751    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"]
Jul 07 00:37:33.259159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259127    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bdcbb93-4747-4452-adab-e782a7244437" containerName="test-curl-7404-139911071281632"
Jul 07 00:37:33.259159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259138    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bdcbb93-4747-4452-adab-e782a7244437" containerName="test-curl-7404-139911071281632"
Jul 07 00:37:33.259159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259148    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy"
Jul 07 00:37:33.259159 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259154    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy"
Jul 07 00:37:33.259299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259166    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy"
Jul 07 00:37:33.259299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259173    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy"
Jul 07 00:37:33.259299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259249    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy"
Jul 07 00:37:33.259299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259256    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bdcbb93-4747-4452-adab-e782a7244437" containerName="test-curl-7404-139911071281632"
Jul 07 00:37:33.259299 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.259261    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf7d58c-fcf7-4522-b132-97e4dcf8d56a" containerName="istio-proxy"
Jul 07 00:37:33.262542 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.262517    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.265897 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.265879    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vhskf\"/\"default-dockercfg-hdd62\""
Jul 07 00:37:33.266017 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.266000    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vhskf\"/\"openshift-service-ca.crt\""
Jul 07 00:37:33.266827 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.266811    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vhskf\"/\"kube-root-ca.crt\""
Jul 07 00:37:33.272931 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.272905    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"]
Jul 07 00:37:33.358249 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.358214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-sys\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.358249 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.358248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-podres\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.358505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.358275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-proc\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.358505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.358295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-lib-modules\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.358505 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.358315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77w4\" (UniqueName: \"kubernetes.io/projected/03c4a508-0371-4110-9579-2494e2d8fa9b-kube-api-access-s77w4\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459620 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-sys\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459620 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-podres\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459715    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-sys\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459734    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-podres\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-proc\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-lib-modules\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.459870 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459830    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77w4\" (UniqueName: \"kubernetes.io/projected/03c4a508-0371-4110-9579-2494e2d8fa9b-kube-api-access-s77w4\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.460142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459870    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-proc\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.460142 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.459889    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03c4a508-0371-4110-9579-2494e2d8fa9b-lib-modules\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.467752 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.467730    2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77w4\" (UniqueName: \"kubernetes.io/projected/03c4a508-0371-4110-9579-2494e2d8fa9b-kube-api-access-s77w4\") pod \"perf-node-gather-daemonset-96shz\" (UID: \"03c4a508-0371-4110-9579-2494e2d8fa9b\") " pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.572483 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.572350    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:33.706815 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.706733    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"]
Jul 07 00:37:33.709540 ip-10-0-141-123 kubenswrapper[2564]: W0707 00:37:33.709511    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod03c4a508_0371_4110_9579_2494e2d8fa9b.slice/crio-32d27d450812b79506574b515a4ed06a2d174ad6275347322fc6cd7844c88cbe WatchSource:0}: Error finding container 32d27d450812b79506574b515a4ed06a2d174ad6275347322fc6cd7844c88cbe: Status 404 returned error can't find the container with id 32d27d450812b79506574b515a4ed06a2d174ad6275347322fc6cd7844c88cbe
Jul 07 00:37:33.903649 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.903618    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/2.log"
Jul 07 00:37:33.910944 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:33.910921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-cht2h_541e45ab-dde2-48dc-b018-b187bcf1387a/console-operator/3.log"
Jul 07 00:37:34.317234 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:34.317198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz" event={"ID":"03c4a508-0371-4110-9579-2494e2d8fa9b","Type":"ContainerStarted","Data":"8527d1dcfb00d712597315c64982813038bb5cbe8dec196ca0d19fe9be98be04"}
Jul 07 00:37:34.317651 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:34.317253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz" event={"ID":"03c4a508-0371-4110-9579-2494e2d8fa9b","Type":"ContainerStarted","Data":"32d27d450812b79506574b515a4ed06a2d174ad6275347322fc6cd7844c88cbe"}
Jul 07 00:37:34.317651 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:34.317281    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:34.334420 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:34.334365    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz" podStartSLOduration=1.3343496369999999 podStartE2EDuration="1.334349637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:37:34.332561117 +0000 UTC m=+2486.745575040" watchObservedRunningTime="2026-07-07 00:37:34.334349637 +0000 UTC m=+2486.747363544"
Jul 07 00:37:34.938338 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:34.938306    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-zbc8b_f059a428-7a21-4caf-a2e5-d0f726632bc9/volume-data-source-validator/0.log"
Jul 07 00:37:36.024028 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:36.023996    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t9ptt_5ef761fc-cc8b-4706-a815-25372d7f87e3/dns/0.log"
Jul 07 00:37:36.042506 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:36.042480    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t9ptt_5ef761fc-cc8b-4706-a815-25372d7f87e3/kube-rbac-proxy/0.log"
Jul 07 00:37:36.064224 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:36.064193    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b95gn_ecb0f702-58cc-4ffd-b2df-f480b889d76d/dns-node-resolver/0.log"
Jul 07 00:37:36.735795 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:36.735763    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wsjmn_02710aba-44b6-4d11-8cc3-3e0d2cb50934/node-ca/0.log"
Jul 07 00:37:37.668398 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:37.668364    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678w6qh2_6609b6e5-bb86-4870-ae30-f1d6a11a661d/istio-proxy/0.log"
Jul 07 00:37:37.912220 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:37.912189    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-x88l5_cdb0be87-65eb-40b6-9c56-e07ddee42b88/kube-auth-proxy/0.log"
Jul 07 00:37:37.941582 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:37.941439    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/0.log"
Jul 07 00:37:37.996203 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:37.996169    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/1.log"
Jul 07 00:37:38.117086 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:38.117052    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kd7mx_ea4d2988-31b5-4bb6-b71f-77554ff8c10e/payload-processing/0.log"
Jul 07 00:37:38.141583 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:38.141552    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68846f8cb5-nx52d_e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726/router/0.log"
Jul 07 00:37:38.896219 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:38.896180    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m2sjv_f57ae867-009e-4fd5-a888-c66d631c614b/serve-healthcheck-canary/0.log"
Jul 07 00:37:39.369643 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:39.369605    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-hkx2q_07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc/insights-operator/0.log"
Jul 07 00:37:39.370294 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:39.370268    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-hkx2q_07b8dee1-5f43-4ef5-bc2f-d83ccbc725cc/insights-operator/1.log"
Jul 07 00:37:39.528922 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:39.528895    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r875p_4e1ba836-686a-46b1-94e4-b5a4fab82637/kube-rbac-proxy/0.log"
Jul 07 00:37:39.548675 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:39.548637    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r875p_4e1ba836-686a-46b1-94e4-b5a4fab82637/exporter/0.log"
Jul 07 00:37:39.570083 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:39.570055    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r875p_4e1ba836-686a-46b1-94e4-b5a4fab82637/extractor/0.log"
Jul 07 00:37:40.332274 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:40.332243    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vhskf/perf-node-gather-daemonset-96shz"
Jul 07 00:37:41.770469 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:41.770430    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rb8zq_a7d5d300-2afb-4e36-b6b7-62846e3aa0fc/manager/0.log"
Jul 07 00:37:42.024705 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.024620    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fpsgs_b3cd4eaf-58f0-4620-931f-6631a3d2d229/manager/0.log"
Jul 07 00:37:42.071758 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.071728    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-9bd647d7b-l86f6_5eb1b25c-e46d-4f2c-9332-38a6c55e5a36/maas-api/0.log"
Jul 07 00:37:42.287527 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.287431    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6966b9487-qhhx6_b4ecc55e-30a9-4b5f-b853-e29cb7ef6785/manager/0.log"
Jul 07 00:37:42.308925 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.308893    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-tsrs9_6c6914c8-2db9-46ad-b6d5-788ec2769ecc/server/0.log"
Jul 07 00:37:42.415312 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.415277    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-f7xn7_0250feb6-d7bc-429e-bde9-d16d3e95ed96/manager/0.log"
Jul 07 00:37:42.559433 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.559360    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-w8xms_fa93f0f7-0145-4ac0-9309-d9fb429247d1/manager/0.log"
Jul 07 00:37:42.580003 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.579981    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcdfc77c6-w8xms_fa93f0f7-0145-4ac0-9309-d9fb429247d1/copy-manifests/0.log"
Jul 07 00:37:42.658274 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:42.658239    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mm8qm_ac916e15-2868-41ec-92b6-9af3ce0d710d/postgres/0.log"
Jul 07 00:37:43.838933 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:43.838903    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678w6qh2_6609b6e5-bb86-4870-ae30-f1d6a11a661d/istio-proxy/0.log"
Jul 07 00:37:44.044376 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:44.044346    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-x88l5_cdb0be87-65eb-40b6-9c56-e07ddee42b88/kube-auth-proxy/0.log"
Jul 07 00:37:44.066066 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:44.066032    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/0.log"
Jul 07 00:37:44.129631 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:44.129586    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/1.log"
Jul 07 00:37:44.238345 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:44.238311    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kd7mx_ea4d2988-31b5-4bb6-b71f-77554ff8c10e/payload-processing/0.log"
Jul 07 00:37:44.252374 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:44.252348    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68846f8cb5-nx52d_e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726/router/0.log"
Jul 07 00:37:45.441924 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:45.441892    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678w6qh2_6609b6e5-bb86-4870-ae30-f1d6a11a661d/istio-proxy/0.log"
Jul 07 00:37:45.657143 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:45.657116    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d7555db7-x88l5_cdb0be87-65eb-40b6-9c56-e07ddee42b88/kube-auth-proxy/0.log"
Jul 07 00:37:45.683831 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:45.683799    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/0.log"
Jul 07 00:37:45.740718 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:45.740644    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kf8gf_7143c880-61d2-423d-a37e-2cb66b2f630f/istio-proxy/1.log"
Jul 07 00:37:45.844295 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:45.844261    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-kd7mx_ea4d2988-31b5-4bb6-b71f-77554ff8c10e/payload-processing/0.log"
Jul 07 00:37:45.861340 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:45.861313    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68846f8cb5-nx52d_e53b48d6-7cc9-4fdf-b6ea-5c5463ce4726/router/0.log"
Jul 07 00:37:49.305073 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:49.305038    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-rppns_637b2feb-ed55-4e15-889e-9d8eb9becc51/kube-storage-version-migrator-operator/1.log"
Jul 07 00:37:49.306556 ip-10-0-141-123 kubenswrapper[2564]: I0707 00:37:49.306519    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-rppns_637b2feb-ed55-4e15-889e-9d8eb9becc51/kube-storage-version-migrator-operator/0.log"